-- Logs begin at Fri 2022-05-27 14:18:39 UTC, end at Fri 2022-05-27 15:32:43 UTC. -- May 27 14:56:54.816651 dv-cinder-842541-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 14:56:54.827525 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 14:56:54 2022] *** May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: nodename: dv-cinder-842541-3-iscsi May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: machine: x86_64 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: clock source: unix May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: pcre jit disabled May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: detected number of CPU cores: 4 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: current working directory: / May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: detected binary path: /usr/bin/uwsgi-core May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: your processes number limit is 47680 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: your memory page size is 4096 bytes May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: detected max file descriptor number: 2048 May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: lock engine: pthread robust mutexes May 27 14:56:54.828055 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: thunder lock: enabled May 27 14:56:54.830103 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 14:56:54.830103 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 14:56:54.873151 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: Python main interpreter initialized at 0x561fe258f9c0 May 27 14:56:54.873151 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: python threads support enabled May 27 14:56:54.873151 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: your server socket listen backlog is limited to 100 connections May 27 14:56:54.873151 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: your mercy for graceful operations on workers is 90 seconds May 27 14:56:54.873484 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: mapped 403077 bytes (393 KB) for 2 cores May 27 14:56:54.873561 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: *** Operational MODE: preforking *** May 27 14:56:54.873839 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: *** uWSGI is running in multiple interpreter mode *** May 27 14:56:54.873839 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: spawned uWSGI master process (pid: 80304) May 27 14:56:54.874034 dv-cinder-842541-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 14:56:54.878375 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: spawned uWSGI worker 1 (pid: 80314, cores: 1) May 27 14:56:54.878375 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: spawned uWSGI worker 2 (pid: 80315, cores: 1) May 27 14:56:54.878375 dv-cinder-842541-3-iscsi devstack@keystone.service[80304]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 14:57:00.544590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:57:00.544590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: from cryptography.utils import int_from_bytes May 27 14:57:00.545397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:57:00.545397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: from cryptography.utils import int_from_bytes May 27 14:57:00.558179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:57:00.558179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: from cryptography.utils import int_from_bytes May 27 14:57:00.558650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:57:00.558650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: from cryptography.utils import int_from_bytes May 27 14:57:00.594976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 14:57:00.595171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 14:57:00.595367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] command line args: [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 14:57:00.595639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 14:57:00.595818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ================================================================================ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 14:57:00.596076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] admin_token = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.596323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_dir = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.596541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.596726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] config_source = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.596907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] debug = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.597106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_config_append = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.598851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_dir = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.599308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.599553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.599804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.600054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.600302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.600500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.600681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.600933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.601180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.601421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.601660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.601896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.602132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.602382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.602626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.602901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.604157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.604513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] publish_errors = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.604761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.604999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.605182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.605432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.605673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.605919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] standard_threads = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.606152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.606397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.606575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.606754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_journal = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.606839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 14:57:00.606935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_json = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.607026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 14:57:00.607109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_stderr = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.607183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] command line args: [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 14:57:00.607259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] use_syslog = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.607443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 14:57:00.607521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.607626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ================================================================================ {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 14:57:00.607740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.607885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] admin_token = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.607969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.608144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] config_dir = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.608222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.608335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.608475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.608568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] config_source = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.608721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] debug = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.608859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.608948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.609043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.609157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.609237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.609346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.609424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.609609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.609689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.609864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.609944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.610099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.610176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.610282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_config_append = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.610367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.610447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.610596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.610680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_dir = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.610792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_file = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.610876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.611031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.611110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.611282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.611369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.611519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.611597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.611770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.611849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.611952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.612042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.612121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.612256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.612367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.612518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.612643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.612773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.612880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.613034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.613112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.613286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.613373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.613536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.613622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.613769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.613848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.614001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.614091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.614233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.614311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.614469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.614719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.614801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.614950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] publish_errors = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.615024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.615179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.615258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.615408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.615486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.615598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.615674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.615851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.615935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.616095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.616184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.616337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] standard_threads = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.616416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.616592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.616671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.616830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.616911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.617192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] use_journal = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.617373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] use_json = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.617580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] use_stderr = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] use_syslog = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.617792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.617951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:57:00.618188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.618960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.619957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.620930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.621877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.622981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.623986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.624999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.625963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.626889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.627991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.628954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.629977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630446 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.630905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.631863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.external = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.632987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.password = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.633997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] auth.token = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.634977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.635968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.636961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.637931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.638926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.639992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.640967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.641920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.642883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.external = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.643853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.password = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] auth.token = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.644852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.645944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.646999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.647962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.648937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.649903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.650622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.650735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.650955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.651971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.652973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.653950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.654990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.655938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.656944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.657880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.658986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.659950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.660921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.661968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.662904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.663865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.664993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.665073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.665241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.665326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.665481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.665595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.666914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.667968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.668965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.669934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.670982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.671882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672118 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.672896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.673878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.674986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.675963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.676938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.677998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] role.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.678962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] role.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.679916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.680913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.681831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.682942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.683945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.684938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.685800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80315) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 14:57:00.685876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.686136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.686387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.686629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.686869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.687105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.687345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.687585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.687835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.688074 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.688333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.688607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.688850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.689099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.689352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.689603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.689844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.690085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.690323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.690571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.690831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.691066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.691312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.691605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.691868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.connection = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.692115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.692365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.692630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.692874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.693112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.693351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.693590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.693829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.694076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.694313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.694557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.694794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.695042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.695280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.695526 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.695758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.695997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.696234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:57:00.696405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80314) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 14:57:00.717660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.717893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.718120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.722466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.722702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.722931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.726765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.733681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.756538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.756772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.760601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.760839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.811067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.811965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.862804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.863040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.864440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.864696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.930954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.931088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.931168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.931340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.932062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.932258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.955954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.956983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.957755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.968891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.969214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.982456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.982723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.982821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.982955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.983905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80315) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:00.984039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80314) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:57:01.002005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.002219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:57:01.002555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.002637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.002756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:57:01.002838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:57:01.003026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.003285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.003391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.003543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.003633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:57:01.003800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.003880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.004093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.004347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.004634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.015729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.016002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.016269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.016572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.016937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.017216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.017488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.017772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.024752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.026205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.028247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.028549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.028828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.029107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.029386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.029542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.029649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.029823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.029904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.030873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.031055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.031144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.031327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.031606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.031880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.032156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.032425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.058433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.060005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.060174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.060533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.061665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.061945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.065793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.066074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.066352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.067273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.067553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.067830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.074409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.074684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.075949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.076232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.079190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.079842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.080126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.080411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.080713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.080992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.081196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.081277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.081483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.081562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.081765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.082048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.082324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.082594 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.082863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.083130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.100324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.100633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.102400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.102678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.106344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.107959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.110033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.110328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.111814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.112108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.115449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.117367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.119662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.121618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.123478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.123779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.124074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.124368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.124674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.124953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.125454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.125755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.126049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.126331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.126626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.126908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.144170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.144463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.144736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.144999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.146173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.146548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.146817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.147083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.153234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.154955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.155171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.155254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.157002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.157289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.158321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.158601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.158881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.159159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.159431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.160374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.160679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.160957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.161234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.161507 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.173606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.173883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.174168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.174448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.174725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.175006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.175287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.175716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.175997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.176280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.176590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.176874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.177151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.177438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.194109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.196355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.197688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.199718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.201039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.202659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.203104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.204303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.205021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.205111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.205389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.206654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.206942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.207239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.213443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.215068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.215388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.217027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.218365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.218660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.218945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.219221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.220326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.220642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.220932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.221217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.228890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.229181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.229564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.230707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.230996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.231280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.237527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80315) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.237806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.238087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.238367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.238637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.238906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.239198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.239327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80314) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:57:01.239463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.239597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.239743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.239878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.240899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80315) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.241952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.242213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.242476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.242741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80314) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:57:01.281354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561fe258f9c0 pid: 80315 (default app) May 27 14:57:01.282914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561fe258f9c0 pid: 80314 (default app) May 27 14:57:01.284599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9ccf4-0a28-422e-89b7-9f4c2787c349 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:01.284893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9ccf4-0a28-422e-89b7-9f4c2787c349 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:01.285171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9ccf4-0a28-422e-89b7-9f4c2787c349 None None] PATH_INFO: `/v3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:01.289470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 791 bytes} [Fri May 27 14:57:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:09.351221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:09.351791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:09.351791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:09.356945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1/2] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:09.362068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:09.362460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:09.362614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:09.431259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: INFO dbcounter [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] Registered counter for database keystone May 27 14:57:09.452975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG oslo_db.sqlalchemy.engines [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80315) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 14:57:09.459406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writer thread running {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 14:57:09.643259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG passlib.handlers.bcrypt [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80315) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 14:57:09.643581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG passlib.handlers.bcrypt [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80315) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 14:57:09.698921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:09.700542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:09.825945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 27 14:57:09.980990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:10.049508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: INFO dbcounter [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] Registered counter for database keystone May 27 14:57:10.069809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG oslo_db.sqlalchemy.engines [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80314) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 14:57:10.077036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writer thread running {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 14:57:10.197880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66438f61-b648-42dc-928a-9d7df7deb62a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:10.203095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:10.203750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:10.204034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:10.204315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:10.212637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:10.266719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 14:57:10.266998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a657ef0-174c-4d12-b048-5f430bcae53a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:10.276944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 2/4] 10.222.0.41 () {58 vars in 1324 bytes} [Fri May 27 14:57:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 14:57:10.282921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:10.297596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebfd01-8207-408d-a08d-5dc6af95a3c5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:10.302072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:10.303200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:10.303491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:10.303772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] PATH_INFO: `/v3/projects/1ab0327c3107460fb9252bf46e1b56cc` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:10.305313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] RBAC: Authorizing `identity:get_project(project_id=1ab0327c3107460fb9252bf46e1b56cc)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:10.465535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 14:57:10.466181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87789c-7095-4eed-ac2f-bd8175ef7a9d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:10.470600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 3/5] 10.222.0.41 () {56 vars in 1207 bytes} [Fri May 27 14:57:10 2022] GET /identity/v3/projects/1ab0327c3107460fb9252bf46e1b56cc => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:12.653610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb7-6f28-4dde-8ea6-fc5ab2f591e4 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:12.654106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb7-6f28-4dde-8ea6-fc5ab2f591e4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:12.654106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb7-6f28-4dde-8ea6-fc5ab2f591e4 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:12.658266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 3/6] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:12.663612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:12.664039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:12.664178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:12.683469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:12.685012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:12.692807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 4/7] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 14:57:12.715331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71cfb7-6f28-4dde-8ea6-fc5ab2f591e4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:12.727457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71cfb7-6f28-4dde-8ea6-fc5ab2f591e4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:12.732438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:12.733096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:12.733383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:12.733662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:12.741358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:12.743153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e7a1-143f-4a94-b116-3430fd596263 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:12.752120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 4/8] 10.222.0.41 () {58 vars in 1324 bytes} [Fri May 27 14:57:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 14:57:12.757483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:12.766539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f54a6-e20c-4a46-8b46-e654487b6994 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:12.770937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:12.771795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:12.772081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:12.772362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:12.773780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:12.775616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50298-bfc5-438c-a642-d7edcb0fbd1f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:12.778044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 5/9] 10.222.0.41 () {56 vars in 1198 bytes} [Fri May 27 14:57:12 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:15.068584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1646-d2de-44c3-86d7-88dbdc9e75d6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.069124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1646-d2de-44c3-86d7-88dbdc9e75d6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.069124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1646-d2de-44c3-86d7-88dbdc9e75d6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.070563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 5/10] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:15.076491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.077229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.077229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.094557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:15.095723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:15.157123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260aebe-a2ee-4a64-98c7-e4ca49e6048c None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.157405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260aebe-a2ee-4a64-98c7-e4ca49e6048c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.157691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260aebe-a2ee-4a64-98c7-e4ca49e6048c None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.159030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 6/11] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:15.163922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.164212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.164525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.184217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 6/12] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:15.205480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG passlib.handlers.bcrypt [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80314) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 14:57:15.205904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG passlib.handlers.bcrypt [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80314) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 14:57:15.207881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.232579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:15.254218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:15.255825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:15.316608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fd389-9137-4238-897f-b6761bae9fb9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.353316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.354040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.354324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.354605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.361770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 7/13] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:15.375935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:15.385636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:15.386796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:15.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:15.388741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.388819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 7/14] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:15.392570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.495355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:15.516023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e9d95-3f3e-4e88-a975-31c968d019e6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.519557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc6a95-1077-4178-bfd5-dc6f4f5f2686 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.522054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:15.559838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.560549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.560838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.561121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.561884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:15.562198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:15.563466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.563654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:15.564375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.564698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.564980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.576898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 8/15] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:15.582861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.655624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:15.657169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:15.688440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516473b-9f10-4130-9b01-919e657f3d26 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.731415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.732307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.732622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.732900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.734416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:15.736365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:15.738037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 9/16] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:15.743515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.759014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:15.773571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 8/17] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:15.780613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.783528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:15.856284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8ad4a-deec-4e82-a0c2-88b52c23ddef None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.894181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900fe1c-61e3-417f-9b27-3e0d73b71f0e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:15.896057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.896917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.897203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.897485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.898685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:15.900180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:15.903601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 10/18] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:15.908041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.935619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:15.936956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:15.937436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:15.937901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:15.958494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:15.959998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:57:15.982460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:15.983384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:15.984275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:57:15.984275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:57:15.984275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:57:15.984275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:15.984527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 9/19] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:15.993569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:15.999439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:16.024003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8be68-dbcb-4ec1-ab6d-eb3c2b6b2c17 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:16.057196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:16.064834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:16.065502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:16.065788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:16.066075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:16.066914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.domain.id=default, user.id=bcba1f2f592c4d6cbf88e82920040619) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:16.067229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:16.068736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940afa3-e8d5-47dc-a37e-a4565762e2e0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:16.075941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 11/20] 10.222.0.41 () {56 vars in 1322 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/role_assignments?user.id=bcba1f2f592c4d6cbf88e82920040619&scope.domain.id=default&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:16.110775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0abf7a-dd1d-4b9b-980f-cd566c0b4ea2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:16.148384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:16.149081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:16.149417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:16.149701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:16.150450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:16.150761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:16.152242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7577b3-5e22-4382-9eb2-c84bdc84215a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:16.172327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 10/21] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:19.031231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367cf9-fd8b-448e-84e2-d5a389055f2a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.031768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367cf9-fd8b-448e-84e2-d5a389055f2a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.031842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23367cf9-fd8b-448e-84e2-d5a389055f2a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.033393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 12/22] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:19.039125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3742ec-f3c7-420e-b255-74abbfe2a32d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.039627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3742ec-f3c7-420e-b255-74abbfe2a32d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.039725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3742ec-f3c7-420e-b255-74abbfe2a32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.066326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2f3742ec-f3c7-420e-b255-74abbfe2a32d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:19.067567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2f3742ec-f3c7-420e-b255-74abbfe2a32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:19.201315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e3db-d10c-4f72-8cce-f1254bbd135a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.201643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e3db-d10c-4f72-8cce-f1254bbd135a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.201883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948e3db-d10c-4f72-8cce-f1254bbd135a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.203451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 13/23] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:19.209554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.209847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.210131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.245020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:19.246403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:19.307985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 11/24] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:19.309240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f68ee-a791-45ff-a0cc-d2c4ad154dd3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.309535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f68ee-a791-45ff-a0cc-d2c4ad154dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.309812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f68ee-a791-45ff-a0cc-d2c4ad154dd3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.311267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 12/25] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:19.316817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.317108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.317396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.354641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:19.356416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:19.403587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 14/26] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:19.404154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:19.452812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:19.504236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 13/27] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:19.504861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:19.534983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:19.563831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a24a2-8c7f-4806-aa6c-cca554886b66 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:19.599912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:19.601072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.601519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.601946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.613879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:19.615515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:19.632526 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12227dd-be50-419c-8b8b-5b97fdd6b8dd None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-447e5337-cdae-45e9-9ef2-fc2f4a8d0969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:57:19.635724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:19.636518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:19.637831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:57:19.637831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:57:19.637831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:57:19.637831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:19.640084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 15/28] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:19.641579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464dc8-65a1-478c-ab82-72629d5b16fc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.641863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464dc8-65a1-478c-ab82-72629d5b16fc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.642140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3464dc8-65a1-478c-ab82-72629d5b16fc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.647955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 16/29] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:19.648852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464dc8-65a1-478c-ab82-72629d5b16fc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:19.675120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:19.675827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.676076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.676356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.677279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:19.678726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8f04a-8e45-48d0-88ae-59de28d37c53 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:19.763275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3464dc8-65a1-478c-ab82-72629d5b16fc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:19.790742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:19.801981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:19.803155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.803656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.804221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.815753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:19.819094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 14/30] 10.222.0.41 () {60 vars in 1131 bytes} [Fri May 27 14:57:19 2022] POST /identity/v3/roles => generated 219 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:19.825639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:19.826551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:19.827288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:19.827866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.827866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.827866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.829956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 17/31] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:19.831018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:19.862371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:19.863959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:19.942240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:19.952990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df359033-1bbb-49e6-ba16-d17f9a6b92af None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:19.994142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:19.995370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:19.997330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:19.997831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:19.998975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:20.000969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.002675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.039802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 18/32] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:20.044047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.054687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 15/33] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:20.056220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.115831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.177100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff314c6d-bca4-4c40-9345-d6b57c1d8d10 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.198272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aba94c-cff6-45f4-8399-3315c51a461a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.218211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.219330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.219852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.220346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.221375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:20.221887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.223757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.235305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 19/34] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:20.236562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.245165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.245165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.245426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.245698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.246998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.248867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.289206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 16/35] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:20.290077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.318414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.382488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20001f9d-6c87-4ae9-b1dd-ae3ba39d7b37 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.395780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.446615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.447312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.447651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.447938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.448808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.450798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e4aecc-5f72-4986-bcbe-c6d9c37b6fbd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.467223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b30fae-4de2-4186-99dc-5b031b0b435a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.509735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.510636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.510926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.511208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.512796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.514285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69af2cf-0446-4180-b209-7851fc451125 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.516199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 17/36] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:20 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:20.519843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5751-354f-4dda-a2e6-427a172ca6ec None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.519843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5751-354f-4dda-a2e6-427a172ca6ec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.519843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb5751-354f-4dda-a2e6-427a172ca6ec None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.519843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 18/37] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:20.524558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e240-4cc7-4ff8-ada4-ea6f3c4f4249 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.524859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e240-4cc7-4ff8-ada4-ea6f3c4f4249 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.525146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e240-4cc7-4ff8-ada4-ea6f3c4f4249 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.526713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 19/38] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:20.527379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1e240-4cc7-4ff8-ada4-ea6f3c4f4249 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.587080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 20/39] 10.222.0.41 () {60 vars in 1131 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/roles => generated 225 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:20.588413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.588754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.589042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.613913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:20.615604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:20.631783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.651123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1e240-4cc7-4ff8-ada4-ea6f3c4f4249 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.690048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.690884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.691153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.691438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.692817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.694325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4fc32-51c2-4782-8b41-1bdc0c3c776f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.700940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 20/40] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:20.700940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.700940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.700940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.711963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 21/41] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:20.713786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.717812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:20.719622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:20.798678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 21/42] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:20.799550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.837975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.843947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1c2dd-c632-42ca-be35-245926f23ac7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.880712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.881591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.881889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.882162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] PATH_INFO: `/v3/domains/default/users/bcba1f2f592c4d6cbf88e82920040619/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.884208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bcba1f2f592c4d6cbf88e82920040619, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.885857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.895555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:20.902269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 22/43] 10.222.0.41 () {58 vars in 1361 bytes} [Fri May 27 14:57:20 2022] PUT /identity/v3/domains/default/users/bcba1f2f592c4d6cbf88e82920040619/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:20.903389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:20.922584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74938831-e544-4a72-bcd1-207e3eb606f1 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:20.963839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:20.964727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:20.965962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:20.966421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:20.967788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:20.969356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:20.971104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 22/44] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:20.976837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:21.032236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7987793-de1b-4fa4-8307-ba979a9f6546 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:21.075998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:21.077120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:21.077415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:21.077700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:21.078465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:21.080754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0303d-58db-468d-8e58-e068de2293d9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:21.098411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a020d8f-4d48-451d-8a1d-f33144378725 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:21.100905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 23/45] 10.222.0.41 () {60 vars in 1131 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:21.139081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:21.139771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:21.140055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:21.140332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:21.141096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:21.142589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f7a01-c766-45bf-b633-f361801e3677 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:21.175697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 23/46] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:57:20 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:57:23.147393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1a9b-c4b3-4428-ae89-23c303c144f3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.147914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1a9b-c4b3-4428-ae89-23c303c144f3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.147997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1a9b-c4b3-4428-ae89-23c303c144f3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.149350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 24/47] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:23.155432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfd097-2525-4b1b-ba6e-fadcec5bff1c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.155840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfd097-2525-4b1b-ba6e-fadcec5bff1c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.156071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfd097-2525-4b1b-ba6e-fadcec5bff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.162047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.174156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-dccfd097-2525-4b1b-ba6e-fadcec5bff1c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:23.175346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-dccfd097-2525-4b1b-ba6e-fadcec5bff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:23.248409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 24/48] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:23.271745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe1a9b-c4b3-4428-ae89-23c303c144f3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.291108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.392267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe1a9b-c4b3-4428-ae89-23c303c144f3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.394638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217b70-a565-462f-9adf-7c2bceb88bda None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.394935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217b70-a565-462f-9adf-7c2bceb88bda None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.395220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217b70-a565-462f-9adf-7c2bceb88bda None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.396640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 25/49] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:23.401558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.401798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.402083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.418833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:23.420039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:23.428699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.429497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.429789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.430069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.440786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:23.448390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:23.449318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:23.450709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 25/50] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:23.456143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.490981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 26/51] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:23.514205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.533930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.555450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f133c-13f1-4405-a6eb-c9c582cac2a3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.576781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.591851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.592546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.592842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.593127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.593876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:23.594189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:23.595596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:23.603853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 26/52] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:23.609736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.615603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1664a-563e-4538-b2d3-bae85f0638ca None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.649723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.650521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.650810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.651097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.652326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:23.653920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:23.655583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 27/53] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:23.661138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.719585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6b72-01c6-4019-9cff-926c95e54faa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.754132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.755009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.755296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.755584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.756874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:23.758391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:23.759957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 27/54] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:23.763678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82894a-1d14-4083-a8f5-4af1b9fa1e58 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.765824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:23.776479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.794287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:23.799533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.800197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.800516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.800807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.801528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:23.802983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:23.828269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 28/55] 10.222.0.41 () {60 vars in 1140 bytes} [Fri May 27 14:57:23 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:57:23.870959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d3dc0-279d-4f65-8635-699663f83650 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:23.908201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:23.908958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:23.909276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:23.909560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:23.910819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:23.912345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cd390-513a-4f1f-bf71-333a46e97ca7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:23.914304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 28/56] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:23.920974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:24.025916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43472c1a-499a-4f50-8caa-726d3e994570 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:24.061889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:24.062556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:24.062846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:24.063129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:24.063977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.domain.id=default, user.id=bcba1f2f592c4d6cbf88e82920040619) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:24.064297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:24.065791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd3a70-3a08-4d67-ba5e-c273bbfe74ef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:24.076366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 29/57] 10.222.0.41 () {56 vars in 1322 bytes} [Fri May 27 14:57:23 2022] GET /identity/v3/role_assignments?user.id=bcba1f2f592c4d6cbf88e82920040619&scope.domain.id=default&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:26.041526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7b509-8fd2-4da3-a01a-6c9735feb7d4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:26.042081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7b509-8fd2-4da3-a01a-6c9735feb7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:26.042081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7b509-8fd2-4da3-a01a-6c9735feb7d4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:26.043492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 29/58] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:26.048707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f273-a049-4444-af06-075b97392c26 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:26.049109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f273-a049-4444-af06-075b97392c26 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:26.049286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f273-a049-4444-af06-075b97392c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:26.066410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f084f273-a049-4444-af06-075b97392c26 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:26.067606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f084f273-a049-4444-af06-075b97392c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:26.111706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:26.135807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 30/59] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:26.159934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7b509-8fd2-4da3-a01a-6c9735feb7d4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:26.237574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:26.260905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7b509-8fd2-4da3-a01a-6c9735feb7d4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:26.293047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:26.294318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:26.294607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:26.294881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:26.296048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:26.297550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e0a47-349f-473f-9bd8-079ca5607502 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:26.299149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 30/60] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:26.304606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084f273-a049-4444-af06-075b97392c26 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:26.400122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084f273-a049-4444-af06-075b97392c26 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:26.433264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:26.433934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:26.434409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:26.434690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:26.435400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:26.436944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50229ded-3d5e-4327-9186-7021b8d63bcd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:26.462326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 31/61] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:26 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:29.864081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017afae-41cd-4268-be78-234af663dd8f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:29.865124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017afae-41cd-4268-be78-234af663dd8f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:29.865633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017afae-41cd-4268-be78-234af663dd8f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:29.867209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 31/62] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:29.873347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:29.874234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:29.874711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:29.899426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:29.900963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:29.939965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:29.961855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b6fc-8aa1-428a-bb39-dc5a132d18d7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:29.962169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b6fc-8aa1-428a-bb39-dc5a132d18d7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:29.962423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16b6fc-8aa1-428a-bb39-dc5a132d18d7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:29.963861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 32/63] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:29.970638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:29.970975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:29.971210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.002086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:30.003298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:30.028549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 32/64] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:30.052504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.138020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 33/65] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:30.173578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.209912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e3426-9ecb-4894-80a4-cc3c6458f661 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.261233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.263891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.264397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.264851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.269274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.285806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9ed67b39-e846-4e2a-a708-762efdae9dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:30.297284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:30.298231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:30.299093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 33/66] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:30.299965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2cc85-ac10-4044-b3ff-cab03f24a46a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.300406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2cc85-ac10-4044-b3ff-cab03f24a46a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.300870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2cc85-ac10-4044-b3ff-cab03f24a46a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.302288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 34/67] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:30.306554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fb5ec-43ec-4ed2-a532-a093b8172892 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.307547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2cc85-ac10-4044-b3ff-cab03f24a46a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.335201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.350754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.351568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.351859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.352142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.365144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0877778c-d133-4701-b851-e160b9813d37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:30.373089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:30.373930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:30.374698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:30.378983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 34/68] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:30.380383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070673a-79f6-40a6-a036-b46fc536eccc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.380706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070673a-79f6-40a6-a036-b46fc536eccc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.380991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070673a-79f6-40a6-a036-b46fc536eccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.398834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a070673a-79f6-40a6-a036-b46fc536eccc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:30.400122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a070673a-79f6-40a6-a036-b46fc536eccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:30.428930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2cc85-ac10-4044-b3ff-cab03f24a46a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.464390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.465096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.465414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.465699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.467041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:30.467362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:30.468800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d37f42-d222-4f3b-ad57-86be3b791f79 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:30.474656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 35/69] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:30.476061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9350-7625-46d5-a9bb-60dae2cc68f9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.476361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9350-7625-46d5-a9bb-60dae2cc68f9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.476669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a9350-7625-46d5-a9bb-60dae2cc68f9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.478134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 36/70] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:30.478716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a9350-7625-46d5-a9bb-60dae2cc68f9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.484785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 35/71] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:30.486456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b88dd-9e8d-4a86-a667-28c8f640018a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.486939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b88dd-9e8d-4a86-a667-28c8f640018a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.487406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b88dd-9e8d-4a86-a667-28c8f640018a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.488877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 36/72] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:30.494162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.494565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.494848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.512077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:30.513920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:30.546292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.592474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 37/73] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:30.592474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.597676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a9350-7625-46d5-a9bb-60dae2cc68f9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.599621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.633404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.634075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.634362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.634638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.635387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:30.635704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:30.637140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786fc2e-06f8-40f5-8008-886a1c83154c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:30.646138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 37/74] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:30.647310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.647604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.647884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.663962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:30.665100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:30.683822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4afc1-40e9-4076-bb5c-f005edf99eb9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.718304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.719176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.719463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.719742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.720955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:30.722408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:30.724039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 38/75] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:30.724835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.731827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 38/76] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:30.733172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.788894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.808564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:30.824580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86ef8a-a0fd-4198-95c1-e69730ed89f5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.834127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a74ea8-e7b5-4d05-9673-46d54a8dbce4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.858888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.859692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.859974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.860257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.868455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:30.869259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:30.869548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:30.869840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:30.870098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:30.878175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:30.879493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:30.880245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 39/77] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:30.881122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:30.884772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:30.889240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:30.890057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:30.890822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 39/78] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:30.891450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:30.986655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9508db-1de7-4270-a447-6c70bcaef388 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:30.992020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6659333-5510-4761-a7fc-6315ef9a0f4a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.007999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.020283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.021185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.021467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.021738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.026797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.027453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.027744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.028029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.037547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.039024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.040729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 40/79] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.041490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.044228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.102010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.103510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.142202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f8966-c1b6-4ee9-b634-87a8658a8d8e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.176576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.177372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.177660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.177945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.179137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.180634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.182257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 41/80] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.183023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.204860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 40/81] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:31.209836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.225354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.257317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.288015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13100e8-deb9-49ac-9ed0-31285b030b89 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.306439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc342b-6242-4236-b351-c6d07305a452 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.322428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.323099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.323392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.323679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.324424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:31.324765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.326179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.334643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 42/82] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.335356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.344259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.344989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.345281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.345567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.346324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:31.346639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.348034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.356200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 41/83] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.360595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.440394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.443278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa668-f195-4d5a-a475-9f10cd2fca60 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.462556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17d019-f811-4bdf-92fe-053e9f528d1c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.472644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.478175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.478827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.479116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.479401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.496855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.497556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.498026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.498317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.499027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.500534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.526691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 42/84] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:31 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:31.527317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.554670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.556184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.646605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e3f43-30a2-473a-936e-1739deea5697 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.671008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 43/85] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:31.671871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.693156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.694556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.694858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.695145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.696649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.698171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.705306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 43/86] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.705306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.708148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.709545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:31.804925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cce2d9-0d42-4a59-a049-8618d738ef1f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.835352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec10704-91e3-4dd1-a599-90e75a63419b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:31.851937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.853773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.853773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.854196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.855626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.857897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.859795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 44/87] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.860543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.881127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:31.881637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:31.881637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:31.882092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:31.883534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:31.885061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:31.886867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 44/88] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:31.888607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:31.969250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.000555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.003348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdaafb-3cb9-4144-a3c6-ad107b2656c4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.006979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271420e-d29e-49b8-901c-d3fefb843468 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.044047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.045155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.045739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.046213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.047243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=bcba1f2f592c4d6cbf88e82920040619) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:32.047755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.049470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.050417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.051293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.051581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.051864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.053102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.054572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.056527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 45/89] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:32.057637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 45/90] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:31 2022] GET /identity/v3/role_assignments?user.id=bcba1f2f592c4d6cbf88e82920040619&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:32.058639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:32.068555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:32.210336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c7401-6230-45b3-9f3b-f6c5dc57c9cd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.251718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.252794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.253279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.253781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.300433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.302851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae2b7c-cd88-4029-a42c-a09cd71b05d6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.346621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.347338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.347633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.347915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.359107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.375721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.376967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.465525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.468170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.564425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 46/91] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:32.565632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:32.631376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 46/92] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:32.632155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:32.680881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.702770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:32.759355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b38f9b-21b8-47ae-aa12-e934d25c417f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.812033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6d25-2995-4d96-8240-34f6d9b91110 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:32.824681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.825419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.826199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.826480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.827320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:32.827638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.829129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.843896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 47/93] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=66fc02452abf40a398e393da3430e5d2 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:32.845853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:32.862586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:32.863521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:32.863810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:32.864090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:32.865526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:32.867063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:32.868886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 47/94] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:32.881641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:33.039833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce37a3e-69ec-44d9-b7d1-0f36274c4c16 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:33.076358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:33.088604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:33.090000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:33.091443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:33.097097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] PATH_INFO: `/v3/projects/1576293e11c240088f85bbce079721d3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:33.098649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] RBAC: Authorizing `identity:get_project(project_id=1576293e11c240088f85bbce079721d3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:33.100394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:33.105897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 48/95] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/projects/1576293e11c240088f85bbce079721d3 => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:33.111853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:33.139270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73220ec-e664-4c93-b916-8c055514f548 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:33.141382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:33.225163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:33.226276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:33.226789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:33.227288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:33.228342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:33.228930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:33.230618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa4e4-5493-4295-9aef-40fc97000598 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:33.243345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 48/96] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:32 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:33.326509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c3b31-d3e0-4ef1-9c20-3b6eb16252fc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:33.363492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:33.364163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:33.364466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:33.364761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:33.365621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=1576293e11c240088f85bbce079721d3, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:33.365943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:33.367437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ffd26-59eb-4e7e-bacd-3563ac2027db None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:33.374349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 49/97] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:33 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=1576293e11c240088f85bbce079721d3&role.id=66fc02452abf40a398e393da3430e5d2 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:34.288354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7ba1-c22b-437c-83c9-8eefa3440f03 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.289191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7ba1-c22b-437c-83c9-8eefa3440f03 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.289191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da7ba1-c22b-437c-83c9-8eefa3440f03 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.290617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 49/98] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:34.297042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.297805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.297805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.324457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:34.325759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:34.414817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:34.425039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 50/99] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:34.460620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da7ba1-c22b-437c-83c9-8eefa3440f03 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:34.602431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da7ba1-c22b-437c-83c9-8eefa3440f03 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:34.651318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:34.652129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.652415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.652719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.669850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e664f375-95eb-4811-84c8-d22b510e9c1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:34.677655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:34.678538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:34.679314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 50/100] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:34.688141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:34.841022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b2d7-091f-4892-8c83-f0efb8317ed3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:34.855402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d390631-0ec2-4949-9bfc-d5fbd42ddfb4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.855774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d390631-0ec2-4949-9bfc-d5fbd42ddfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.856069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d390631-0ec2-4949-9bfc-d5fbd42ddfb4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.857549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 51/101] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:34.874070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.874371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.874644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.891143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:34.891832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:34.892168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:34.892492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:34.893246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:34.893556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:34.894973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:34.904129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 51/102] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:34.909478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:34.917472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:34.918782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:34.984794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:35.001597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:35.101798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 52/103] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:35.106437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beacef-3bee-456b-b55f-cb06ee502708 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:35.158534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb500167-454c-4002-a225-48e13b82417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:35.159423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500167-454c-4002-a225-48e13b82417f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.159709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500167-454c-4002-a225-48e13b82417f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.159994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500167-454c-4002-a225-48e13b82417f None admin] PATH_INFO: `/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.170885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:35.181896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb500167-454c-4002-a225-48e13b82417f None admin] RBAC: Authorizing `identity:get_user(user_id=cfeac4cc269b4042b6ae9bcc3adf714f)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:35.184131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb500167-454c-4002-a225-48e13b82417f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:35.186580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 52/104] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:34 2022] GET /identity/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:35.193285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb500167-454c-4002-a225-48e13b82417f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:35.429508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb500167-454c-4002-a225-48e13b82417f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:35.469670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:35.520303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f0e8d-0305-4e68-b861-5d4f4fcc44a5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:35.529317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:35.530011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.530309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.530598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.609165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:35.610241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.610691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.611120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.612525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:35.614147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:35.616229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 53/105] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:35.625581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:35.677447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:35.688494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:35.690175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:35.820648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b285-219d-4599-924d-49d84f0d6471 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:35.887464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:35.896488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.896488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.896488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.896488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:35.896488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f97e8-3b31-4008-9fbc-8a254fe447fe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:35.943702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 54/106] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:57:35 2022] POST /identity/v3/projects => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:57:35.946225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17fbc-3df8-445c-a68f-f12457cfdbe7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.946719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17fbc-3df8-445c-a68f-f12457cfdbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.947166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17fbc-3df8-445c-a68f-f12457cfdbe7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.949579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 55/107] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:35.957896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:35.959698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:35.960155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:35.974097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 53/108] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 780 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:57:35.997360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:36.026833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:36.033169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:36.052307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:36.265189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599740c7-95ed-4a21-becc-65a764d084d0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:36.281770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 56/109] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:36.321355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:36.336876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:36.344488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:36.344726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.344726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.344726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.344726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:36.346733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:36.351094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 54/110] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:35 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:36.363455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:36.543836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:36.583347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72455e3e-dbc5-4c14-a83d-ba0345ab253c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:36.614048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b40a24-61b9-4ea8-9c48-ee1d455484d7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:36.624627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:36.626230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.627122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.627760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.629690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=cfeac4cc269b4042b6ae9bcc3adf714f) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:36.630313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:36.633435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7c20-e1dd-453f-8b66-0d2e01d82f43 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:36.645397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 55/111] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:36 2022] GET /identity/v3/role_assignments?user.id=cfeac4cc269b4042b6ae9bcc3adf714f&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:36.647876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8605f-7928-4026-b321-61df05646048 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.648489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8605f-7928-4026-b321-61df05646048 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.649206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8605f-7928-4026-b321-61df05646048 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.651836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 56/112] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:36.657599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a2a45-3f55-49ac-b9cd-6d49d21e6190 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.657599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a2a45-3f55-49ac-b9cd-6d49d21e6190 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.657599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a2a45-3f55-49ac-b9cd-6d49d21e6190 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.659345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 57/113] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:36.665559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.666083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.666690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.668504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:36.669703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.670205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.670760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.683757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d45b0791-2c4b-4607-a84c-ddaddb355e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:36.693237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:36.694122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:36.694913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:36.697976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 57/114] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:36.699965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9096181-8930-40c2-84d9-645031663630 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.700467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9096181-8930-40c2-84d9-645031663630 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.701466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9096181-8930-40c2-84d9-645031663630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.703828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:36.705244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:36.734068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d9096181-8930-40c2-84d9-645031663630 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:36.735320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d9096181-8930-40c2-84d9-645031663630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:36.843931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 58/115] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:36.845094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:36.876467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:36.890997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 58/116] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:36.892348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d72c-be01-435d-b11f-f1828de0c35f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:36.892664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d72c-be01-435d-b11f-f1828de0c35f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:36.892946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3068d72c-be01-435d-b11f-f1828de0c35f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:36.894209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 59/117] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:36.894897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d72c-be01-435d-b11f-f1828de0c35f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:36.960026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:36.996870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5050-96ca-4eac-8237-b7ab3e6f3e2a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.018629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3068d72c-be01-435d-b11f-f1828de0c35f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.044664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.045363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.045673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.046105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.046869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:37.047181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.049189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566b183-218e-4089-997a-118e3506fbfd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.056406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.057632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.058122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.058657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.064035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 59/118] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.065816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.066110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.066392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.074677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:37.083825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:37.085211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:37.086164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:37.086908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:37.087950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:37.088712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 60/119] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:37.089839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.156966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.177192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 60/120] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:37.178164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.219465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a8625-fdf3-44d0-919b-6d2ff5ffb8b7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.264660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.267119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.267928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.268213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.268513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.285889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:37.296287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:37.297168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:37.297901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 61/121] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:37.299188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.311520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170c717-2f30-46a9-a323-a0462b2ad5d3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.351763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.352734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.353021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.353350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.354637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.356031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.358368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 61/122] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.362958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.430982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a814d-7330-47c0-83c7-7934eafa0818 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.460840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.476216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.476983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.477435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.477863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.478701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:37.479023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.480964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.483469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5d340-0289-4bad-9f35-b9545d2043c1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.491871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 62/123] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.492877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.520756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.521974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.522458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.522933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.526167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.537266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:37.545997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:37.547302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:37.548071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:37.550454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 62/124] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:37.556084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.619359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3430-9231-49af-a4e8-cce31cbe01fd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.661109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.662252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.662757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.663268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.664309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:37.664850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.666474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.676616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 63/125] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.677713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.679927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e4316-ef01-4254-a7b6-b2c62449a21c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.728951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.729250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.730128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.730562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.730712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.732799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.736310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.739072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 63/126] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.740429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.801914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086d4d-a4e7-420c-ba92-7546fa18e7ce None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.827601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:37.840647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.841020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.841429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.841748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.842981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.844492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.846500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 64/127] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.847209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.881646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2174db-8830-48d5-bfd3-0c28302495e0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.923955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:37.924791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:37.925195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:37.925553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:37.926381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:37.926823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:37.928426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:37.937578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 64/128] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:37.938395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:37.966046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f9f37-3ed2-4661-a2ab-da45dcaa0eac None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:37.976838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.005170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.006690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.006982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.007267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.009186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.010970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.013363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 65/129] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:37 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:38.014157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.056844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315fc2e-ca14-4b35-b159-c4b2462728ee None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.086510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.108678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.109698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.109968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.110255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/users/bcba1f2f592c4d6cbf88e82920040619/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.112400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, user_id=bcba1f2f592c4d6cbf88e82920040619, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.113840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.130922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 65/130] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:37 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/users/bcba1f2f592c4d6cbf88e82920040619/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:38.132013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.135369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b170-4a0e-4cea-beef-f94c681c1c0f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.176011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.177264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.177785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.178284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] PATH_INFO: `/v3/projects/1576293e11c240088f85bbce079721d3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.179863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] RBAC: Authorizing `identity:get_project(project_id=1576293e11c240088f85bbce079721d3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.181817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.184124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 66/131] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:38 2022] GET /identity/v3/projects/1576293e11c240088f85bbce079721d3 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:38.186093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.266233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.288184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f747b1-a1dc-44f2-aa15-2125ce1f0cdc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.370842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.371956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.372348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.373340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.381346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.383075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.385031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 66/132] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:38 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:38.385338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462f9e3-3766-4e0a-aeef-9f236122eb8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.385969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.447093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.448123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.448418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.448836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.450066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.450667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.452573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.454441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 67/133] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:38 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:38.457975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.522834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af16a85-5033-457f-8359-cb7c5a917c76 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.557953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.558848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.559131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.559408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] PATH_INFO: `/v3/projects/1576293e11c240088f85bbce079721d3/users/f3c65b39372942929f6e010bdbdf024b/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.562000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] RBAC: Authorizing `identity:create_grant(project_id=1576293e11c240088f85bbce079721d3, user_id=f3c65b39372942929f6e010bdbdf024b, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.563476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.577585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc330d-1de1-4465-bc50-76c009cd9d09 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.593903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.600809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 67/134] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:38 2022] PUT /identity/v3/projects/1576293e11c240088f85bbce079721d3/users/f3c65b39372942929f6e010bdbdf024b/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:38.601914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.636292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.641477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.641769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.642046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.646412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.649255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.655471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 68/135] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:38 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:38.662143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:38.848802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8942bb-b069-4378-834f-2830a8f85273 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.874730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:38.902041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda4b11-4112-4f1e-9ddc-be7a69c54ad9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:38.909180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.910134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.910495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.910782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/users/f3c65b39372942929f6e010bdbdf024b/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.912965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, user_id=f3c65b39372942929f6e010bdbdf024b, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.914548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634152d-93c7-4cb2-9ad1-7a60e892b960 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.946669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 68/136] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:38 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/users/f3c65b39372942929f6e010bdbdf024b/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:38.946974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:38.947989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:38.948283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:38.948637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/users/f3c65b39372942929f6e010bdbdf024b/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:38.951269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, user_id=f3c65b39372942929f6e010bdbdf024b, role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:38.952960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d93b0-6134-4b45-a945-583414e2c960 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:38.980601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 69/137] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:38 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/users/f3c65b39372942929f6e010bdbdf024b/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:39.018411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beef30-9e8a-4e45-875e-0427c4d75b54 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.018725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beef30-9e8a-4e45-875e-0427c4d75b54 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.018981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beef30-9e8a-4e45-875e-0427c4d75b54 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.020402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 69/138] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:39.026752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.027064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.027317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.046836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:39.048069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:39.088218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:39.133360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 70/139] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:39.172400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beef30-9e8a-4e45-875e-0427c4d75b54 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:39.306148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beef30-9e8a-4e45-875e-0427c4d75b54 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:39.344360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:39.345251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.345539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.345820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.347100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:39.348642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d7341-1f6b-413d-9fc8-867d57ac6e4e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:39.350367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 70/140] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:39.356110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:39.481634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8716e-bfab-4c2c-87ca-0e39d3752d23 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:39.521845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:39.522568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.522844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.523124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.523861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:39.525418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c34a18f-5a86-4069-9d96-65b23b0e8aa8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:39.557091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 71/141] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:39 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:39.565551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db1c99-71d0-44a4-af2c-1aaba0411008 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.566164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db1c99-71d0-44a4-af2c-1aaba0411008 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.566164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db1c99-71d0-44a4-af2c-1aaba0411008 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.567555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 71/142] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:39.573631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.574297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.574791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.598726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:39.600288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:39.632762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:39.694883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 72/143] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:39.741970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db1c99-71d0-44a4-af2c-1aaba0411008 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:39.806999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:39.887178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db1c99-71d0-44a4-af2c-1aaba0411008 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:39.925170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:39.925980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:39.926266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:39.926547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:39.945623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:39.953608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:39.954524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:39.955684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 72/144] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:39.962768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:40.111911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862db185-e4b9-45f8-b415-3674d768aa9b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:40.155279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:40.155921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:40.156210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:40.156512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:40.157281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:40.157595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:40.159010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:40.169045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 73/145] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:40.175830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:40.315790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01dd81-b5f6-4258-a5ad-b6cbb02c7255 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:40.317423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:40.357736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:40.358629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:40.358919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:40.359196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] PATH_INFO: `/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:40.360985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] RBAC: Authorizing `identity:get_user(user_id=cfeac4cc269b4042b6ae9bcc3adf714f)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:40.362454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:40.364430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 73/146] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:40 2022] GET /identity/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:40.370114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:40.403139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:40.495072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbb43-b090-458a-8ad1-4b7305bab630 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:40.536174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:40.537090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:40.537375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:40.537663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:40.538872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:40.540349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1fed-307f-48c4-83ef-d8c05e714fc5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:40.542313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 74/147] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:40 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:40.548990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:40.672486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb91e2e-2909-4cda-830d-14dcf41eb800 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:40.712260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:40.713196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:40.713484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:40.713760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/users/cfeac4cc269b4042b6ae9bcc3adf714f/roles/969fc1ce10964e99ac25ca5ebc9437a9` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:40.715746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, user_id=cfeac4cc269b4042b6ae9bcc3adf714f, role_id=969fc1ce10964e99ac25ca5ebc9437a9)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:40.717341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48f93-6187-4274-a99a-81ba93600146 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:40.741203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 74/148] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:40 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/users/cfeac4cc269b4042b6ae9bcc3adf714f/roles/969fc1ce10964e99ac25ca5ebc9437a9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 14:57:41.039805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513390-245b-423e-baf8-ac587e2d4bcc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.040302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513390-245b-423e-baf8-ac587e2d4bcc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.040302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4513390-245b-423e-baf8-ac587e2d4bcc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:41.041708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 75/149] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:41.047451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.047826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.048116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:41.069961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:41.071167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:41.103244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:41.157714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 75/150] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:41.184133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4513390-245b-423e-baf8-ac587e2d4bcc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:41.281404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:41.299102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4513390-245b-423e-baf8-ac587e2d4bcc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:41.339104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:41.340010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.340299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.340622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:41.356623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:41.364974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:41.365826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:41.367005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 76/151] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:41.372048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:41.502727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818debe-3576-43e8-ad82-d6871bd2307f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:41.541125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:41.541808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.542097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.542371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:41.543126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:41.543435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:41.545595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:41.560788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 76/152] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:41.565896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:41.714238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc58e8-141f-4f90-91ee-e7a792c4d324 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:41.752948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:41.753898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.754185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.754463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] PATH_INFO: `/v3/users/bcba1f2f592c4d6cbf88e82920040619` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:41.761172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] RBAC: Authorizing `identity:get_user(user_id=bcba1f2f592c4d6cbf88e82920040619)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:41.762679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:41.764472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 77/153] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:41 2022] GET /identity/v3/users/bcba1f2f592c4d6cbf88e82920040619 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:41.769975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:41.814247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:41.916388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd2576-694f-4bc2-82de-9775d31a67e2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:41.959462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:41.960385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:41.960696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:41.960975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.047925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:42.049432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:42.078282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:42.160837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 77/154] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:42.170335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:42.199852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:42.322846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e2f4b-a435-4631-a25c-ecdc1f558aad None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:42.368300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:42.369124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:42.369411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:42.369695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.370912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:42.372380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52264e7-b73a-4026-afd0-9fdda707806b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:42.374752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 78/155] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:42 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:42.383605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:42.554944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0bc1-9f88-4166-a483-8a8784bda812 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:42.555173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cbbfd-7556-494e-8b5b-122191e214f9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:42.555711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cbbfd-7556-494e-8b5b-122191e214f9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:42.556220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cbbfd-7556-494e-8b5b-122191e214f9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.557839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 79/156] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:42.563907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:42.564527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:42.565030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.591168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:42.593544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:42.594310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:42.617352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:42.618034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:42.618375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:42.618599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.619451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=bcba1f2f592c4d6cbf88e82920040619) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:42.619766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:42.621357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:42.635591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 78/157] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:42 2022] GET /identity/v3/role_assignments?user.id=bcba1f2f592c4d6cbf88e82920040619&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:42.712832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 80/158] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:42.765097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:42.902685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792c683-00cf-4065-b695-5d67469044b9 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:42.945333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:42.946144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:42.946427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:42.946700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:42.963066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:42.971505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:42.972337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:42.973131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 79/159] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:42.979701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:43.003578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:43.126922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51d72e-680c-478d-9860-0e62e1305089 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:43.171953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:43.172562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.172880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.173123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.173876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:43.174183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:43.175576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7dd8b7-18e7-4792-acf3-e643754bb466 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:43.187628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 81/160] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:43.193276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:43.284797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:43.361532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00153f6-7073-4b95-9dc0-60d566d03d30 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.361822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00153f6-7073-4b95-9dc0-60d566d03d30 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.362337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00153f6-7073-4b95-9dc0-60d566d03d30 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.364160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 82/161] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:43.366764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbcf96-b16c-4a05-88c6-7c67e9bc6e30 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:43.372153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd13bff-414b-4063-8acc-d25e9cc9361c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.372739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd13bff-414b-4063-8acc-d25e9cc9361c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.373266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd13bff-414b-4063-8acc-d25e9cc9361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.409004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6fd13bff-414b-4063-8acc-d25e9cc9361c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:43.410516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6fd13bff-414b-4063-8acc-d25e9cc9361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:43.430201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:43.431027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.431312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.431595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.433451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:43.434462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:43.436699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 80/162] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:43 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:43.442045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:43.558825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 83/163] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:43.560164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80229-daf3-48bb-a79a-21a72d1c5642 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.560503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80229-daf3-48bb-a79a-21a72d1c5642 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.560796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80229-daf3-48bb-a79a-21a72d1c5642 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.562220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 84/164] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:43.569198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.569487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.569771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.580814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:43.594736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:43.595937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:43.718829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ccb77-9cd1-4301-b530-c47d19c80fae None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:43.745667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:43.772119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 85/165] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:43.773379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:43.782913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:43.783592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.783882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.784167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:43.865553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:43.867317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:43.944419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c0c6-bca5-4548-9f12-28e546b0414c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:43.988887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 81/166] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:57:43.990488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:43.997749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:43.998702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:43.999243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:43.999610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.011686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:44.017691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:44.026680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:44.027619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:44.028550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:44.038664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 86/167] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:44.039038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:44.045643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:44.213083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a2cb2-0dd2-4885-9b7b-aec5fca8a12b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:44.278586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:44.280341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.280867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.281400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] PATH_INFO: `/v3/projects/1576293e11c240088f85bbce079721d3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.283225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] RBAC: Authorizing `identity:get_project(project_id=1576293e11c240088f85bbce079721d3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:44.285264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:44.287482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 87/168] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/projects/1576293e11c240088f85bbce079721d3 => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:44.288437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:44.348252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f9b95-3a66-42f0-b8d7-498def892c69 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:44.364810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:44.399755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434ad72-9249-4918-9ca2-b501efc7a6ab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:44.425760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:44.426874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.427376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.427846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.449999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:44.452646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:44.453586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.454070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.454524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.455472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:44.455923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:44.457482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1fe4e-1599-44e3-9798-2a2add6f04e0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:44.464659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:44.465764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:44.466725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:44.468638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 82/169] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 478 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 88/170] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56714cb-91ae-457a-b8f2-5c675902b39d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56714cb-91ae-457a-b8f2-5c675902b39d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56714cb-91ae-457a-b8f2-5c675902b39d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 89/171] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:44.475269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56714cb-91ae-457a-b8f2-5c675902b39d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:44.476127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:44.610447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:44.631212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56714cb-91ae-457a-b8f2-5c675902b39d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:44.655019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a11bb-1188-4f34-8575-a2da3b7cfd1d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:44.678717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:44.700568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:44.702556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.703736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.704435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.706042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:44.706753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:44.709046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:44.723835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:44.725102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.727117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.728054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.731642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 90/172] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:44.732871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=1576293e11c240088f85bbce079721d3, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:44.733401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:44.736140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:44.737262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b5ed8-f9bc-4b2b-b7bc-99e5be9aac87 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:44.750197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 83/173] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=1576293e11c240088f85bbce079721d3&role.id=66fc02452abf40a398e393da3430e5d2 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:44.751888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:44.752402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:44.753532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:44.784647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:44.786416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:44.917578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 84/174] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:44.919248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.039639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764d226-fe2f-4182-8c3a-86abd211898c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.066083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.109644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e370a-e7ff-4a53-a29f-2c1545f4f6c0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.145866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.146749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.147034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.147302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.148653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.148845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.149567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.149858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.150143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] PATH_INFO: `/v3/users/f3c65b39372942929f6e010bdbdf024b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.150326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.151394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] RBAC: Authorizing `identity:get_user(user_id=f3c65b39372942929f6e010bdbdf024b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.152205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 85/175] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:45.153067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.153172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da45ca-26eb-4beb-9586-ea0c30708f48 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.155069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 91/176] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:44 2022] GET /identity/v3/users/f3c65b39372942929f6e010bdbdf024b => generated 275 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:45.156270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5626fb-d9ea-4b91-8524-56f790b6aff6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.156585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5626fb-d9ea-4b91-8524-56f790b6aff6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.156869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5626fb-d9ea-4b91-8524-56f790b6aff6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.158133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 92/177] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:45.165635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.165926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.166211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.188933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:45.190068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:45.246638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.269018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 93/178] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:45.270017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.271353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568098bf-17c6-4614-b2d3-ae1836a9a306 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.306672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.307561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.307873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.308174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.311536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.320076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:45.328365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:45.329238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:45.330003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 86/179] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:45.330630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.376360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00b46d-6dc0-4016-98e7-3eda17e1bd35 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.414220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.414879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.415162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.415440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.443707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14e9de-7820-4e7e-a39c-2974580f1b33 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.469514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.479239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.479888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.480174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.480471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.485755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.487263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de4456-51e9-4a86-81a3-33ce6be9827d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.542387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.549833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.551325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.586199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 94/180] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:45.587319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e733-4dc0-40e7-95b6-09f4da0735f8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.587608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e733-4dc0-40e7-95b6-09f4da0735f8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.587887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e733-4dc0-40e7-95b6-09f4da0735f8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.589329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 95/181] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:45.590256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fbae-dc64-46ce-b0f8-8b8e4564fba4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.590868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fbae-dc64-46ce-b0f8-8b8e4564fba4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.591156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1fbae-dc64-46ce-b0f8-8b8e4564fba4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.596513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 96/182] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:45.596513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1fbae-dc64-46ce-b0f8-8b8e4564fba4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.654396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 87/183] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:45.655252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.681520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.698462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1fbae-dc64-46ce-b0f8-8b8e4564fba4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.733093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.733881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.734161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.734432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.735602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.737108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.738730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 97/184] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:45.742055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.752349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.758729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09877a7-f5e8-40fb-809a-16ac420dbc51 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.792808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.793478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.793765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.794054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.794878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:45.795343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.797051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48397b7f-5642-4df7-8d9d-63f4cb0f8845 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.805556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 88/185] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:45.806875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.807186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.807489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.825029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:45.826222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:45.842723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efd7b8-38e4-4c28-8e7b-9c3ddf55f1b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:45.877219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:45.878083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.878365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.878640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.879825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:45.881314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eac7d9-08c3-44a1-b9c8-ee409a66149c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:45.883010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 98/186] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:45.884134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:45.884421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:45.884724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:45.905686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 89/187] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:45.905686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:45.905937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:45.905937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:45.917213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:45.984558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 99/188] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:45.984558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.008585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.020895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209479f6-73a2-427d-92ae-90fbd25a8449 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.056985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.057842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.058121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.058394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.059561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.061955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.063735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 90/189] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:45 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:46.064461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.079601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046808a6-67ca-4e56-9d4a-2fc803972f0b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.115845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.116606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.116894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.117175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.117887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.119410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.139120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 100/190] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:45 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:46.140379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.149260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.179402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e09d7c-3a23-4141-b1f7-c4d7eb3e8ecc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.219135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.220040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.220352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.220662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.237828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.239708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.241738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 91/191] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:46.242824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.260588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.261555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb14bf8-dc52-4761-871a-aade43f52faa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.301782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.302484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.302767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.303038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.303891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:46.304200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.305747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.313356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 101/192] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:46.314212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.402747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53693af-b079-4494-bdd4-9dcbe115b7ee None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.418127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.460955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.462092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.462523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.462969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.471072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b141e-8354-49b4-941f-e5676a5e67e0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.478371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:46.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:46.490992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:46.491870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:46.495824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 92/193] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:46.495824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.519791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.520869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.521221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.521559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.538552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:46.547480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:46.548611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:46.549568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:46.550966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 102/194] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:46.551841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.594474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.622780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc9bc6-5cf0-46e1-a1b6-26b0fcfd5565 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.669286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.671616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.671845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.672211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.673857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=f3c65b39372942929f6e010bdbdf024b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:46.674178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.675956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.676832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509db9-4648-49c2-b0d6-e36c87f491da None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.683789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 93/195] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/role_assignments?user.id=f3c65b39372942929f6e010bdbdf024b&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=66fc02452abf40a398e393da3430e5d2 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:46.685180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.738291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.755993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.756619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.757005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.757291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.846328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.847951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.868694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f4f1a-68c9-43c4-9c89-fcff20906e5e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:46.933770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:46.934349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:46.934856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:46.935441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:46.937065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:46.937606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:46.939285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:46.948798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:46.959381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 94/196] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:46.960850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:46.963622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 103/197] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:46.964963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.003990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.095743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d9ca-8e01-4b5b-b012-487d7a5f100d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.136982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a015a-aae4-4486-b9a1-1f8a85e5114a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.142276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.143222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.143507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.143782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] PATH_INFO: `/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.145145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] RBAC: Authorizing `identity:get_user(user_id=cfeac4cc269b4042b6ae9bcc3adf714f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.146625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.148743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 104/198] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/users/cfeac4cc269b4042b6ae9bcc3adf714f => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.149585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.177410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.178088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.178367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.178644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.179500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:47.180018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.181369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.191658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 95/199] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.192542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.261555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.265190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.271264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f1284-8f95-448b-94d4-a1988bd242ea None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.307163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869ae5e-b4ca-43ad-9952-ec8c758f6cc8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.310059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.311566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.311779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.312054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.313467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.315082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.316925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 105/200] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.317695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.344766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.345430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.345712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.346044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.419231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.421344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.430010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b60baa-9eb1-4679-ac3c-f65cb675bc03 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.468860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.469744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.470194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.470477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.471869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.473350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.480028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 106/201] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.481113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.486295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.519735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.531312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 96/202] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:47.532323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.596611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6b19c-6de7-49b9-804e-038e8f163e2e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.636396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.637192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.637472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.637743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.638570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=ac928a8589d94f3bb5825cfd56820f61) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:47.638905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.640092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785d1d6-05da-4ebf-a387-62340b67d248 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.640417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.646711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 107/203] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/role_assignments?user.id=ac928a8589d94f3bb5825cfd56820f61&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.647619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.677202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.677978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.678266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.678540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.733460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.801061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb967625-82eb-4fcf-a516-c4bc70e63f0f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:47.802846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:47.830636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.832148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.843284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:47.844230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:47.844543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:47.844818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:47.846080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:47.847635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:47.849784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 108/204] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:47.858684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:47.977326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 97/205] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:47.985440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:48.023573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f2afb-3ac6-4af0-b9d1-5f8506f6841a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:48.065572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:48.066263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.066547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.066833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.076574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, scope.project.id=22887df109864ec39f82aaf569743a97, user.id=cfeac4cc269b4042b6ae9bcc3adf714f) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:48.076574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:48.076574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:48.076574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 109/206] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/role_assignments?user.id=cfeac4cc269b4042b6ae9bcc3adf714f&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:48.141715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:48.213483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf93264-6228-498f-86c7-b94ca3f07118 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:48.270334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:48.271232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.271514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.271787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.273031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:48.274501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65c646-452f-4bc9-9995-2d8dfa1b5248 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:48.277217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 98/207] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:47 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:48.282740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:48.320226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:48.399710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737c330-3d2e-4ed5-9223-ac25729a853c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:48.435645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:48.437000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.437419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.437700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.438549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=ac928a8589d94f3bb5825cfd56820f61) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:48.438866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:48.440356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58789a-a073-4ee4-9769-72c4445bd5f7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:48.446886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 110/208] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:48 2022] GET /identity/v3/role_assignments?user.id=ac928a8589d94f3bb5825cfd56820f61&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:48.944602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df2-7187-4226-a170-c86f2b15cb5b None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.945099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df2-7187-4226-a170-c86f2b15cb5b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.945613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f49df2-7187-4226-a170-c86f2b15cb5b None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.947204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 99/209] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:48.953526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.953808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.954238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.973632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:48.975089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:48.985674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbafc-067d-46ff-ac3a-e306f1d90dbf None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.986292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbafc-067d-46ff-ac3a-e306f1d90dbf None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.986776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dbafc-067d-46ff-ac3a-e306f1d90dbf None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:48.989104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 100/210] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:48.994456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:48.994745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:48.995401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.017420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:49.018945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:49.051818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 111/211] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:49.074914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.094151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 101/212] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:49.104094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:49.119443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.184955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:49.198237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541acff-f299-4650-8e5b-0ef84a741e21 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.230176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9d4e-55c4-4748-a9f1-8d6d7602d267 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.236392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.237399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.237825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.238238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.251105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:49.261746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:49.262823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:49.263814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:49.266876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 112/213] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:49.275930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.276086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.276086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.276545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.277099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.278409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:49.280490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:49.282664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 102/214] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:49.288433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.382930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:49.407142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c389-6a76-470f-8b7f-4c1256e77793 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.425739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a12d02-c53e-4ce1-b865-17a32d3c9f3a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.445263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.446051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.446425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.446781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.447599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:49.447980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:49.449563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:49.457368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:49.459233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 113/215] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:49.464511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.472840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.473824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.474331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.474808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.475878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:49.477717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:49.503593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 103/216] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:49 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:57:49.598072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab366b-74b9-4495-9db6-5b0dc2037007 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.639469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.640403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.640804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.641082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] PATH_INFO: `/v3/users/497caa51510941ea99f416cffa4f6325` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.658833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] RBAC: Authorizing `identity:get_user(user_id=497caa51510941ea99f416cffa4f6325)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:49.660408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:49.662544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 114/217] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:49 2022] GET /identity/v3/users/497caa51510941ea99f416cffa4f6325 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:49.668222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:49.780206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c1ff3-d835-4ef2-9cb8-0a81a37ec257 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:49.816903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:49.817917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:49.818425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:49.818899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:49.880745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:49.895272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:49.897095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae026de-4df0-4fe8-a54a-2469d272b03f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:50.010871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 104/218] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:50.017875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:50.020402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:50.138035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c99b-09f4-43fa-b3b8-bc009dd23310 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.138367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c99b-09f4-43fa-b3b8-bc009dd23310 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.138611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c99b-09f4-43fa-b3b8-bc009dd23310 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.140355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 105/219] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:50.145486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.145783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.146070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.167296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9872b986-26e2-4e49-b49a-70a26d5d963d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:50.189581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:50.190818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:50.204018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:50.204917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.205200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.205468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.206686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:50.208172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:50.210160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 115/220] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:50 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:50.215861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:50.291380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:50.303168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 106/221] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:50.325417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5fcf7-f3c6-4b4c-9280-cbd87139aac4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:50.332342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:50.364135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:50.364817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.365104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.365381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.366218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=497caa51510941ea99f416cffa4f6325) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:50.366529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:50.367992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:50.369958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:50.374721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 116/222] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:50 2022] GET /identity/v3/role_assignments?user.id=497caa51510941ea99f416cffa4f6325&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=66fc02452abf40a398e393da3430e5d2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:50.534132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae18629-6805-4fe9-a45e-d6141387ac17 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:50.582398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:50.583207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.583491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.583767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.617395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:50.625970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:50.626879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:50.628078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 107/223] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:50.633255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:50.751340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a4bac-b039-4b1c-9e72-779132577f0e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:50.787559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:50.788156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.788438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.788737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.789484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:50.789791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:50.791193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d45-070c-4a3a-87a6-69d540c8d4b5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:50.793615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:50.801284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 117/224] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:50.806650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:50.910825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:50.947769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b136d-6034-474a-a05c-e0dae9efe85b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:50.989316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:50.990575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.991051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.991513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.993064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:50.993772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a584d-4bca-40e3-81d6-0fb9c50c8f86 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:50.994064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a584d-4bca-40e3-81d6-0fb9c50c8f86 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:50.994342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a584d-4bca-40e3-81d6-0fb9c50c8f86 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:50.995326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f97e58-f9c0-46fd-bc63-a3dab6e9c07f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:50.995969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 118/225] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:50.997828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 108/226] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:50 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:51.004315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a584d-4bca-40e3-81d6-0fb9c50c8f86 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:51.007666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.008161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.008700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.053616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:51.054864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:51.114596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a584d-4bca-40e3-81d6-0fb9c50c8f86 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:51.171897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:51.172947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.173383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.173677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.177233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:51.178718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:51.180663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 119/227] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:51 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:51.189471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:51.200747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 109/228] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:51.222244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:51.322158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:51.346855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:51.393207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3ffa-d6b2-4968-ab75-5454e15a8fb5 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:51.414421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e09052-b650-412d-9869-1996bd7fdcc8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:51.428649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:51.429468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.429750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.430087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.442700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:51.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:51.452176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:51.453553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 110/229] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:51.460047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:51.537170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:51.538091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.538380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.538664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/ac928a8589d94f3bb5825cfd56820f61/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.545849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, user_id=ac928a8589d94f3bb5825cfd56820f61, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:51.547326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:51.570232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d657c8-3c41-4bee-a40c-cff21e97004e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:51.591671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 120/230] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:51 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/ac928a8589d94f3bb5825cfd56820f61/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:51.612860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:51.619892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:51.620523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.620866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.621153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.621909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:51.622218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:51.624240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:51.642910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 111/231] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:51.649617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:51.845658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64eee3c-2150-4adb-8228-06233ca24740 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:51.878576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:51.890802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:51.891894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:51.892181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:51.892473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:51.893718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:51.895197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:51.896916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 121/232] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:51 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:51.902630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:52.019228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a7cb0-3aa7-48ff-95c8-e02277eb2fa1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:52.056141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:52.057059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.057339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.057612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.058809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:52.060342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46b1c1-b948-4706-b560-f4e5f11023f9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:52.062998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 112/233] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:51 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:52.068356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:52.205054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236037d4-f740-4d74-8209-f2ae2748d0e9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:52.255216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:52.256124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.256412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.256719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/ac928a8589d94f3bb5825cfd56820f61/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.259006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, user_id=ac928a8589d94f3bb5825cfd56820f61, role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:52.260519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d847b9-ea3e-4a3a-bd0e-4f5c3cf39d94 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:52.283933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 122/234] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:52 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/ac928a8589d94f3bb5825cfd56820f61/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:57:52.458232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc766e-7d08-4f99-b0d0-0028ec2d3bdd None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.458745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc766e-7d08-4f99-b0d0-0028ec2d3bdd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.458745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc766e-7d08-4f99-b0d0-0028ec2d3bdd None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.460191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 113/235] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:52.466274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29e79-636c-4e4d-b3df-2c9d85da93e1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.466564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29e79-636c-4e4d-b3df-2c9d85da93e1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.466848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29e79-636c-4e4d-b3df-2c9d85da93e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.493494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c7d29e79-636c-4e4d-b3df-2c9d85da93e1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:52.494693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7d29e79-636c-4e4d-b3df-2c9d85da93e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:52.559574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:52.581769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 123/236] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:52.591807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e06f0-af8e-48a0-abde-0f14966f63b6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.592100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e06f0-af8e-48a0-abde-0f14966f63b6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.592572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e06f0-af8e-48a0-abde-0f14966f63b6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.594261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 114/237] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:52.601888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.602182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.602465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.614009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e06f0-af8e-48a0-abde-0f14966f63b6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:52.623145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:52.624998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:52.708918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:52.727855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 124/238] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:52.757843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e06f0-af8e-48a0-abde-0f14966f63b6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:52.771010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:52.814578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:52.815428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:52.815716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:52.815993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:52.834902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:52.850641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:52.851554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:52.852332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:52.852967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 115/239] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:52.862980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:52.969136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6b64e-f9d9-4872-8d2f-eca01c80f0d5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.001348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.044555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.045370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.046223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.046504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.047874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.049407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.051075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 125/240] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:53.057292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.068409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721358c-4da7-43de-9b1e-43fada5ab967 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.098434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.108814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.110412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.110710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.110997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.111764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:53.112083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.113625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.136847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 116/241] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:53.143185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.233757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ab452-40e6-4af9-ae04-f3e714b6c3df None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.270164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.270840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.271183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.271470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.272229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.272563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0ff9-1c47-4f47-b161-9f1433ce0e81 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.273735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c46072-c6bc-4d6b-964e-a165600c0094 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.297922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 126/242] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:53 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:57:53.300668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d717-ac15-41d7-aa91-686f857d65be None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.301022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d717-ac15-41d7-aa91-686f857d65be None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.301755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7d717-ac15-41d7-aa91-686f857d65be None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.303917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 127/243] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:53.310357 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.310839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.311775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.314761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.315654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.315995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.316282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] PATH_INFO: `/v3/users/81e9e1a275824a5dade44452122361cd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.332271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] RBAC: Authorizing `identity:get_user(user_id=81e9e1a275824a5dade44452122361cd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.333939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:53.334089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.335765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:53.336218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 117/244] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/users/81e9e1a275824a5dade44452122361cd => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:53.346456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.381551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.427246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 128/245] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:53.446958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.476790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.558030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241642b-d67e-42d3-aa5a-1f373d36591f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.623255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.624014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.624300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.624651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.638187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ee665-f777-41d4-a6f5-14a5eaf8ed08 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.690686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.691508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.691791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.692065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.708216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:53.709777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.711521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:53.717832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:53.718714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:53.719700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 129/246] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:53.726101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.766295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.769570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:53.840718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 118/247] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:53.846918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.871074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20c75f-ae76-4422-b4e3-f3282b39f3fe None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:53.919541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:53.921096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:53.921441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:53.922180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:53.923425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:53.924012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:53.925628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:53.940214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 130/248] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:53.946504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:53.991663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada69d1a-f5f7-471d-8d18-6abdb295ca6c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.034839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.035819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.036103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.036378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.037708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.039226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.041145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 119/249] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:54.051472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.058316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:54.066317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:54.068854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a764354-3d9e-49bc-85a0-200ebe86b524 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.107548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.109091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.109557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.110017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] PATH_INFO: `/v3/users/497caa51510941ea99f416cffa4f6325` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.111444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] RBAC: Authorizing `identity:get_user(user_id=497caa51510941ea99f416cffa4f6325)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.113956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.116098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 131/250] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:53 2022] GET /identity/v3/users/497caa51510941ea99f416cffa4f6325 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:54.121073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.169409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a3e07-a3bd-4d9f-8d3d-38a11590d963 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.208773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.209490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.209772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.210091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.211233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=81e9e1a275824a5dade44452122361cd) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:54.211477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.212970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fab47-459d-4e72-9fb9-f96a3b75aea6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.219383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 120/251] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:54 2022] GET /identity/v3/role_assignments?user.id=81e9e1a275824a5dade44452122361cd&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:54.230778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5a274-0242-43fc-be76-92a328feee6d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.247563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1028-191e-4222-bf7e-dfa18cee85c6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.247854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1028-191e-4222-bf7e-dfa18cee85c6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.248136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc1028-191e-4222-bf7e-dfa18cee85c6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.252911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 121/252] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:54.258285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.258577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.258857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.271937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.273088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.273522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.274023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.275397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.277049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.278943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 132/253] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:54 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:54.284907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.289022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:54.290262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:54.337020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:54.422492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 122/254] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:54.452334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.462906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:54.464016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2e2c-ebff-40ab-9668-0fb28820a412 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.533925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.534850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.535137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.535417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/497caa51510941ea99f416cffa4f6325/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.542043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, user_id=497caa51510941ea99f416cffa4f6325, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.543541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.574915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 133/255] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:54 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/497caa51510941ea99f416cffa4f6325/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 14:57:54.629675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cf3f7-6b4c-4522-8f72-0e0058c70d3d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.671911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.672964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.673242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.673514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.685452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:54.695777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:54.697930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:54.699572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 123/256] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:54.708363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.840812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:54.855602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bcb52-133f-4141-a802-793da244f26f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:54.905401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:54.906392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:54.906671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:54.906948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:54.907702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:54.908011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:54.909450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:54.921170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 134/257] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:54.927272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:54.998001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:55.047354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b76bd1-db57-4360-8bed-9d011add2041 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:55.089467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:55.090347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.090626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.090898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.092097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:55.094048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb4ad2-f8cf-4837-bcdc-663ce07c0298 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:55.099930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 124/258] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:54 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:55.102037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:55.263325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da06b4-f28c-4442-9a06-fcd3d7f83856 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:55.268319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a0a86-7945-44b2-88d5-fb2cb787b1a8 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.268938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a0a86-7945-44b2-88d5-fb2cb787b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.269438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a0a86-7945-44b2-88d5-fb2cb787b1a8 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.271263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 125/259] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:55.276840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.277406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.277878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.296003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:55.297671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:55.329235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:55.330532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.332002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.333320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.387366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:55.447508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 126/260] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:57:55.467984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:55.469872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:55.527193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:55.640982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:55.651516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 135/261] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:55.654061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:55.709398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354afebe-2aa4-4c3a-b900-32d568d63cf8 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:55.737444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:55.767992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:55.769104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.769565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.770041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.780755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:55.789072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:55.789935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:57:55.790713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:55.793445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 127/262] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:57:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:55.801369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:55.853143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f8834-9243-4e27-ad44-5b4c8c80cff8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:55.938407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:55.939714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:55.940001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:55.940279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:55.944304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:55.945971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:55.952555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 136/263] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:55 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:55.955174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:56.070320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e5404-92df-4de6-82e7-3e469bd0a96f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:56.114901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:56.135178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2baad-f146-443a-ab8f-3c922b14f396 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:56.141310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:56.142257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:56.142754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:56.143211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:56.144135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:56.144644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:56.146224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:56.158874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:56.165361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 128/264] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:57:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:56.169575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:56.197367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:56.198355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:56.198873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:56.199347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:56.200395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=ac928a8589d94f3bb5825cfd56820f61) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:56.200933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:56.203264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:56.215929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 137/265] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:55 2022] GET /identity/v3/role_assignments?user.id=ac928a8589d94f3bb5825cfd56820f61&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:56.337936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a86381-a3b7-49f8-89b8-039a17ea4f15 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:56.399372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:56.400626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:56.401152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:56.401625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] PATH_INFO: `/v3/users/ac928a8589d94f3bb5825cfd56820f61` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:56.405426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] RBAC: Authorizing `identity:get_user(user_id=ac928a8589d94f3bb5825cfd56820f61)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:56.407125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:56.408896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 129/266] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:56 2022] GET /identity/v3/users/ac928a8589d94f3bb5825cfd56820f61 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:56.414480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:56.565704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec158aa6-c3e2-4303-aa8e-bda259b680e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:56.611955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:56.612555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:56.612839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:56.613111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:56.677056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:56.698805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:56.700299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:56.854214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 138/267] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:57:56.860249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:56.947404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.022800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f298f79-935e-4c32-af53-30388fd369f1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.075004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.075907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.076159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.076436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.078441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:57.080004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac295-7279-4a64-a4a7-56a7386b91d8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:57.082003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 130/268] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:56 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:57.087824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.183314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.212491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283f84b-2392-43eb-9204-ebe3bcd86c88 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.214125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68879207-48b8-4a32-a1a8-791a2ab8f42b None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.214621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68879207-48b8-4a32-a1a8-791a2ab8f42b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.215149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68879207-48b8-4a32-a1a8-791a2ab8f42b None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.217005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 131/269] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:57.227023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29242c4f-cbe9-4730-b808-ac17020e28fa None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.227610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29242c4f-cbe9-4730-b808-ac17020e28fa None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.227985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29242c4f-cbe9-4730-b808-ac17020e28fa None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.229915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 132/270] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:57.231284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.231786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.232258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.253541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:57.255407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:57.256835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.257500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.257786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.258063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.258892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=ac928a8589d94f3bb5825cfd56820f61) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:57.259367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:57.261018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46afb6a8-b4e3-4bfe-93da-602334a8ec5c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:57.268598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 139/271] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/role_assignments?user.id=ac928a8589d94f3bb5825cfd56820f61&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:57.270042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.270473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.270893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.289338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:57.290821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:57.356174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 133/272] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:57.372865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 140/273] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:57.399380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.403023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.509246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.551746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.559784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6053a8-a640-428a-8258-d749d49f627c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.588592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3f1ea-4c6d-40b9-b2f2-71da5fe5c8d5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.600886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.601713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.601995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.602433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.604124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:57.605848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:57.608098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 134/274] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:57.615994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.642830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.644008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.645106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.645564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.657607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:57.666435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:57.667327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:57.668099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:57:57.671122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 141/275] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:57:57.676652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.741946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d837635-2f93-4db3-829b-12736266f56c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.774354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.798739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.799409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.799696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.799977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.801040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:57.802690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee053f2-6ff3-40d9-804b-a874415e8cef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:57.812335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698c4bc-8fb6-4e8b-a2ff-23a2e6fdf8f6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:57.830937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 135/276] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:57:57 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:57:57.836704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd987-bee8-44a5-9db5-a2294e981615 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.836704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd987-bee8-44a5-9db5-a2294e981615 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.836704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dd987-bee8-44a5-9db5-a2294e981615 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.836704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 136/277] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:57:57.841473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b500cb-d007-4a4a-a616-eeb544718427 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.841769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b500cb-d007-4a4a-a616-eeb544718427 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.842048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b500cb-d007-4a4a-a616-eeb544718427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.859943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:57.860729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.861051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.861386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.862102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-59b500cb-d007-4a4a-a616-eeb544718427 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:57.862468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:57.862552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:57.863376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-59b500cb-d007-4a4a-a616-eeb544718427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:57.863990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:57.865994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:57.878795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 142/278] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:57:57.884607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:57.949327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 137/279] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:57.971818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541906c7-ed71-40ce-b720-112c7cc06477 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:57.972120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541906c7-ed71-40ce-b720-112c7cc06477 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:57.972498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541906c7-ed71-40ce-b720-112c7cc06477 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:57.976554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 138/280] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:57:57.977473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541906c7-ed71-40ce-b720-112c7cc06477 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.015428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16a7a6-627d-4731-8d84-614293bc2da0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.060144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.060640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.060925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.061206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] PATH_INFO: `/v3/users/e06f0c02c7294526ae78c2f42a47a05c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.081606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] RBAC: Authorizing `identity:get_user(user_id=e06f0c02c7294526ae78c2f42a47a05c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.083101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9babd0d-35a6-4886-9989-98e8ca3af71d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.086804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 143/281] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/users/e06f0c02c7294526ae78c2f42a47a05c => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.086804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.086804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.087057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.102310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541906c7-ed71-40ce-b720-112c7cc06477 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.105055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.110373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:57:58.112138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:57:58.157381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.158204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.158486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.158788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.170186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:58.178821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:58.179829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:58.180746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:58.181950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 139/282] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:58.182595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.208025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 144/283] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:57:58.208917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.303370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a02aa-e07a-4fd0-9011-4b44ce00a507 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.319122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80a422-8cec-4dcc-8b8e-af0eb541ec89 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.342143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.343194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.343533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.343913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.344883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, user.id=e06f0c02c7294526ae78c2f42a47a05c, scope.system=all) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:58.345201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.346708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.352498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 140/284] 10.222.0.41 () {56 vars in 1308 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/role_assignments?user.id=e06f0c02c7294526ae78c2f42a47a05c&scope.system=all&role.id=66fc02452abf40a398e393da3430e5d2 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.353298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.369915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.370993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.371503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.371976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.372994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:58.373514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.375825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.388482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.404116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 145/285] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.413560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.483729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb3fdf-7d20-48fc-9b5a-d4df2a4b921a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.521601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.522427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.522709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.522983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.536148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:57:58.536748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:57:58.544569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:57:58.546460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:57:58.547602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 141/286] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:57:58.558698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.618376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5b7e2-7045-4834-b7ef-d34185df83d7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.664570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.665568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.665852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.666129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] PATH_INFO: `/v3/users/81e9e1a275824a5dade44452122361cd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.673386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] RBAC: Authorizing `identity:get_user(user_id=81e9e1a275824a5dade44452122361cd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.674883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.676752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 146/287] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/users/81e9e1a275824a5dade44452122361cd => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.682072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.688583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.721090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d5fe-5c2d-4579-877c-5f4f07a838dd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.776414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.777109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.777391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.777664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.778411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:58.778714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.780104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.804820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 142/288] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.829157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:58.911222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457e38-214f-40a0-adc3-1d3611ef0d90 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:58.949942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:58.979548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:58.980414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:58.980797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:58.981248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:58.982272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:58.983764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:58.985847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 147/289] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:58.991283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:59.000073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208292-0a44-4890-b60e-fdf218940eaa None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:59.049866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:59.051225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:59.051723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:59.052194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] PATH_INFO: `/v3/users/497caa51510941ea99f416cffa4f6325` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:59.055398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] RBAC: Authorizing `identity:get_user(user_id=497caa51510941ea99f416cffa4f6325)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:59.057294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:59.064089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 143/290] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:57:58 2022] GET /identity/v3/users/497caa51510941ea99f416cffa4f6325 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:59.065739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:59.097040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:59.120776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54babd19-6087-428d-8ff9-0cf688e3fc39 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:59.169470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:59.170283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:59.170565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:59.171412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/81e9e1a275824a5dade44452122361cd/roles/969fc1ce10964e99ac25ca5ebc9437a9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:59.175946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, user_id=81e9e1a275824a5dade44452122361cd, role_id=969fc1ce10964e99ac25ca5ebc9437a9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:59.177453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:59.210246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 148/291] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:57:58 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/users/81e9e1a275824a5dade44452122361cd/roles/969fc1ce10964e99ac25ca5ebc9437a9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 14:57:59.242975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444de09-804b-4e6a-adca-aaa9d3aca039 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:59.293694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:59.294398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:59.294751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:59.295035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:59.397763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:59.399348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:59.428713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:59.523774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 144/292] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:57:59.529680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:59.600755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:59.676710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd8d32-644e-4004-a3eb-84ffe0cfbcfe None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:59.720670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:59.721471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:59.721755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:59.722034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:59.723400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:59.724903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f219c3-dbd7-4bce-acd2-7e7fbf7097c7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:59.726640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 149/293] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:57:59 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:57:59.732554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:57:59.895934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2482c9-f85d-4c1d-8be6-88cea3857956 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:57:59.924048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:57:59.948700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:57:59.949698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:57:59.950205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:57:59.950676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:57:59.951770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=497caa51510941ea99f416cffa4f6325) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:57:59.952382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:57:59.954052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab98e9-51b1-4a70-a40c-ad28bb3b9550 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:57:59.965378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 145/294] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:57:59 2022] GET /identity/v3/role_assignments?user.id=497caa51510941ea99f416cffa4f6325&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=66fc02452abf40a398e393da3430e5d2 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:00.445247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3430-1949-45f9-99db-844b7d0a9fa8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.445752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3430-1949-45f9-99db-844b7d0a9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.445852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b3430-1949-45f9-99db-844b7d0a9fa8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.447170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 150/295] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:00.453899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29477890-06bf-4738-8fd4-a5ce897904d6 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.454948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29477890-06bf-4738-8fd4-a5ce897904d6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.455646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29477890-06bf-4738-8fd4-a5ce897904d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.479904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-29477890-06bf-4738-8fd4-a5ce897904d6 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:00.481491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-29477890-06bf-4738-8fd4-a5ce897904d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:00.565100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 146/296] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:00.592273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b3430-1949-45f9-99db-844b7d0a9fa8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:00.707076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b3430-1949-45f9-99db-844b7d0a9fa8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:00.720594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:00.740529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b149cc-2ff7-422b-897c-3161963f7d0a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.740856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b149cc-2ff7-422b-897c-3161963f7d0a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.741103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b149cc-2ff7-422b-897c-3161963f7d0a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.742751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 147/297] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:00.750604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.750868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.751114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.751745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:00.752703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.753013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.753267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.766006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:00.770792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:00.772008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:00.774034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:00.774902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:00.775679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 151/298] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:00.785524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:00.859911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 148/299] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:00.882466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:00.908535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27d8b6-ba9a-401c-9cde-87addad62acf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:00.931471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:00.947150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:00.948094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.948381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.948691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.949895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:00.950224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:00.952070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50176c87-55c6-4ab7-b61d-9581c362729f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:00.963165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 152/300] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:00.964478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a481-0ce6-4103-82d3-713b60815dbd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:00.964853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a481-0ce6-4103-82d3-713b60815dbd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:00.965132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a481-0ce6-4103-82d3-713b60815dbd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:00.966363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 153/301] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:00.968625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53a481-0ce6-4103-82d3-713b60815dbd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.000622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e13da-60c0-42d9-a96b-0bf7a141c272 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.008033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:01.037934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.038773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.039009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.039288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.040472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.041926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905259e-b71b-497c-afaa-a763294a23cc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.048492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 149/302] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:01.048492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.048492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.048492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.065410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:01.066852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:01.089288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53a481-0ce6-4103-82d3-713b60815dbd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.128378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.129472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.129761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.130044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] PATH_INFO: `/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.143037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 150/303] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:01.143932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.146944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe4d5a4173f49d49e0c4ce014da4a98)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.148517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.150390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 154/304] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:00 2022] GET /identity/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:01.157431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.195892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:01.253818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:01.262524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d3e78-2343-47d5-a7b8-ae2ef69128f2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.263878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29af6-7ac5-45f9-bd24-19895a3f8646 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.299708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.299903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.300380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.300582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.300674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.300872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.300965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] PATH_INFO: `/v3/groups` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.301154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.301617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.301980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, user.id=dfe4d5a4173f49d49e0c4ce014da4a98, scope.system=all) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:01.302296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.303039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.303737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.308961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 155/305] 10.222.0.41 () {56 vars in 1308 bytes} [Fri May 27 14:58:01 2022] GET /identity/v3/role_assignments?user.id=dfe4d5a4173f49d49e0c4ce014da4a98&scope.system=all&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:01.309927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.320000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 151/306] 10.222.0.41 () {60 vars in 1134 bytes} [Fri May 27 14:58:01 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:01.456029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29ca15-2d9b-4c54-b12f-fa818d257ad7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.501148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.501983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.502267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.502543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.516562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:01.524762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:01.525687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:01.527065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 156/307] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:01.533084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.654981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:01.660833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352eb2-3e0c-4645-a29e-46c086a8cdf5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.698569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.699476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.699897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.700310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.701209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:01.701680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.704504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.713862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 152/308] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:01.720965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.738630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:01.835629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ab573-471a-4291-bbf1-9437f9b1d9c4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:01.872125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:01.873029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:01.873310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:01.873582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] PATH_INFO: `/v3/users/e06f0c02c7294526ae78c2f42a47a05c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:01.875321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] RBAC: Authorizing `identity:get_user(user_id=e06f0c02c7294526ae78c2f42a47a05c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:01.876805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99539d00-435f-43c7-9f7e-ec12bb6e8e2b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:01.878497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 157/309] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:01 2022] GET /identity/v3/users/e06f0c02c7294526ae78c2f42a47a05c => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:01.884141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:01.990564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57da20-2e87-4066-b4b7-a0b0af25a93e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:02.000418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62635a0-20e2-431f-8b10-585825fd0258 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.000763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62635a0-20e2-431f-8b10-585825fd0258 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.001078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62635a0-20e2-431f-8b10-585825fd0258 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.006276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 158/310] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:02.007753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.008041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.008322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.025718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:02.026917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:02.028596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:02.029886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.030176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.030992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] PATH_INFO: `/v3/system/users/e06f0c02c7294526ae78c2f42a47a05c/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.032764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e06f0c02c7294526ae78c2f42a47a05c, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:02.034162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:02.044908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 153/311] 10.222.0.41 () {58 vars in 1334 bytes} [Fri May 27 14:58:01 2022] PUT /identity/v3/system/users/e06f0c02c7294526ae78c2f42a47a05c/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:02.120638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 159/312] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:02.146747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:02.180383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:02.270752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ae55c-f202-440e-baf8-8206099ce915 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:02.308141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:02.308970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.309258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.309599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.321160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:02.329501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:02.330372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:02.331156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 154/313] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:02.336993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:02.351590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:02.444742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134cb7a-dd09-4a9a-bea3-da705e3ef235 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:02.481582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:02.483033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.484038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.484541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.485505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:02.486007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:02.487587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:02.496477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 160/314] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:02.502454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:02.605630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:02.623482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017870ee-9e54-40ae-8a80-a7089c6795cf None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:02.660757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:02.661953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.662434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.662903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] PATH_INFO: `/v3/users/81e9e1a275824a5dade44452122361cd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.664331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] RBAC: Authorizing `identity:get_user(user_id=81e9e1a275824a5dade44452122361cd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:02.666007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:02.672557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 155/315] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:02 2022] GET /identity/v3/users/81e9e1a275824a5dade44452122361cd => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:02.674556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:02.755945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:02.784823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824f9c-3200-48a8-9a00-f8452a0966dd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:02.820303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:02.821415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:02.821897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:02.822371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:02.895047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:02.896570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:02.975010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:03.006994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 161/316] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:03.008172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:03.115825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cb7e3-785e-4022-b6cf-d696101dd404 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:03.151570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:03.152462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.152756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.153034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.154233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:03.155757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ac34d-7b40-44f1-81b6-5b23d9ae7a15 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:03.157878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 156/317] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:03 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:03.163563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:03.274834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ffbb4-0cdb-4820-a739-31a67f152c9c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:03.309672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:03.310333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.310614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.310893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.312360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4, user.id=81e9e1a275824a5dade44452122361cd) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:03.312758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:03.314233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f2053-4530-4cab-80ab-163accaa7720 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:03.320803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 162/318] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:03 2022] GET /identity/v3/role_assignments?user.id=81e9e1a275824a5dade44452122361cd&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:03.693779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7007-2405-48cc-9d2f-e832b7550eaf None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.694328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7007-2405-48cc-9d2f-e832b7550eaf None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.694328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf7007-2405-48cc-9d2f-e832b7550eaf None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.695839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 157/319] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:03.700956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.701328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.701516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.715377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04800-d55e-4c1a-bdea-bd4166ef8e3d None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.715673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04800-d55e-4c1a-bdea-bd4166ef8e3d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.715958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f04800-d55e-4c1a-bdea-bd4166ef8e3d None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.717356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 158/320] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:03.720662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:03.722095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:03.722783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.723077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.723360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.740062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:03.741275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:03.760480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:03.761862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:03.808492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 163/321] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:03.828495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 159/322] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:03.828674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:03.848904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:03.939665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661438e-d68a-4f75-9c97-13ce0de43c42 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:03.960838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37af95-3b83-4392-9806-2fc42a2983aa None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:03.976192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:03.977025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:03.977309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:03.977587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:03.988548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:03.996285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:03.997529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:03.998374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 164/323] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:04.004372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.004539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.005251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.005500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.005778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.006952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.008486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.015021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 160/324] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:04.015679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.037550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:04.038130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:04.118393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0fa287-12b0-41d3-8283-38347ac086e7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.123346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d76c4c-16da-4dd0-ad1d-313e2bbca27f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.153169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.153865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.154153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.154433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.155196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:04.156198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.157805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.165417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.166606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.171726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 165/325] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:04.171907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.171907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] PATH_INFO: `/v3/groups` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.171907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.172119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.173758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.200475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 161/326] 10.222.0.41 () {60 vars in 1134 bytes} [Fri May 27 14:58:04 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:58:04.289335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:04.301361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f36aaa-825f-477d-b418-1d0e4157044c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.338350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.339326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.339612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.339894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] PATH_INFO: `/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.341224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] RBAC: Authorizing `identity:get_user(user_id=dfe4d5a4173f49d49e0c4ce014da4a98)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.342708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571daf58-ed10-4587-9a1b-34b6308189ce None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.344876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 166/327] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:04 2022] GET /identity/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:04.350355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.387886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183e24-2fd4-47d3-8cf6-ca7f97549c3f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.388182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183e24-2fd4-47d3-8cf6-ca7f97549c3f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.388488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183e24-2fd4-47d3-8cf6-ca7f97549c3f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.390703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 167/328] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:04.395843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.396133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.396418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.414638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:04.415797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:04.457594 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:04.486899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ad896-4d8a-4b0a-a38b-940f30b385dd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.506849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 168/329] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:04.526317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.527431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.527885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.528318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] PATH_INFO: `/v3/system/users/dfe4d5a4173f49d49e0c4ce014da4a98/roles/969fc1ce10964e99ac25ca5ebc9437a9` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.530023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dfe4d5a4173f49d49e0c4ce014da4a98, role_id=969fc1ce10964e99ac25ca5ebc9437a9)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.531634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.531796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.541304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 162/330] 10.222.0.41 () {58 vars in 1334 bytes} [Fri May 27 14:58:04 2022] PUT /identity/v3/system/users/dfe4d5a4173f49d49e0c4ce014da4a98/roles/969fc1ce10964e99ac25ca5ebc9437a9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:04.643282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:04.659271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf80abb-067a-43a8-b116-df62cb5c5ecf None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.706612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.707442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.707721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.707994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.723159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:04.731086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:04.732022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:04.732880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 169/331] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:04.738746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:04.854303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69dfff-94ad-4769-adf1-2fd39fd90b2f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:04.894277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:04.894970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:04.895254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:04.895533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:04.896291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:04.896626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:04.898031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:04.907549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 163/332] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:04.912875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:05.046002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387fc54-8315-45fa-b3ae-96065355403d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:05.092282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:05.093809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:05.094328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:05.094633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] PATH_INFO: `/v3/users/e06f0c02c7294526ae78c2f42a47a05c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:05.096669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] RBAC: Authorizing `identity:get_user(user_id=e06f0c02c7294526ae78c2f42a47a05c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:05.098387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b85765-77c4-41a8-9cf5-935aaf6d9df6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:05.100702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 170/333] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:04 2022] GET /identity/v3/users/e06f0c02c7294526ae78c2f42a47a05c => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:05.107639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:05.120935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:05.235213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3436eb-c3f7-4cad-b2b9-a66616850c87 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:05.273674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:05.274341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:05.274668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:05.274949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:05.275834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] RBAC: Adding query filter params (role.id=66fc02452abf40a398e393da3430e5d2, user.id=e06f0c02c7294526ae78c2f42a47a05c, scope.system=all) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:05.276153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:05.277630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91eb2a-3950-4561-abb9-07737df0f4e2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:05.290857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 164/334] 10.222.0.41 () {56 vars in 1308 bytes} [Fri May 27 14:58:05 2022] GET /identity/v3/role_assignments?user.id=e06f0c02c7294526ae78c2f42a47a05c&scope.system=all&role.id=66fc02452abf40a398e393da3430e5d2 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:06.983011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb537-13c1-4e3a-951d-8ac72d0f3917 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:06.983522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb537-13c1-4e3a-951d-8ac72d0f3917 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:06.984471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb537-13c1-4e3a-951d-8ac72d0f3917 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:06.985728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 171/335] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:06.992433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640139a-d7a2-4552-92f8-e3c7237a8400 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:06.992923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640139a-d7a2-4552-92f8-e3c7237a8400 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:06.993001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640139a-d7a2-4552-92f8-e3c7237a8400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.013272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7640139a-d7a2-4552-92f8-e3c7237a8400 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:07.014779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7640139a-d7a2-4552-92f8-e3c7237a8400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:07.102437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:07.113081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 165/336] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:07.153862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65deb537-13c1-4e3a-951d-8ac72d0f3917 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:07.182406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:07.302553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65deb537-13c1-4e3a-951d-8ac72d0f3917 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:07.320600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7faf2-9fda-4d80-a80d-579d0b5fb7c6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.320930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7faf2-9fda-4d80-a80d-579d0b5fb7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.321163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7faf2-9fda-4d80-a80d-579d0b5fb7c6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.322811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 166/337] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:07.330609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d86e6-5725-4044-86a9-fdda83943974 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.330895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d86e6-5725-4044-86a9-fdda83943974 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.331176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d86e6-5725-4044-86a9-fdda83943974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.365473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d65d86e6-5725-4044-86a9-fdda83943974 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:07.366669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d65d86e6-5725-4044-86a9-fdda83943974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:07.379255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:07.380107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.380398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.380710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.408698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:07.421053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:07.423000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:07.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:07.426511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 172/338] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:07.431815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:07.539178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 167/339] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:07.540350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4b8a-99b8-49f9-96c1-98c45f3fb909 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.540663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4b8a-99b8-49f9-96c1-98c45f3fb909 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.540940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4b8a-99b8-49f9-96c1-98c45f3fb909 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.542210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 168/340] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:07.549331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.549632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.549900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.565925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:07.576208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:07.577351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:07.605034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3582b49-8202-449b-89dd-b829126e1eb0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:07.662052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:07.662732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.663014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.663291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.664041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:07.664349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:07.667807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:07.673967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 169/341] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:07.675099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:07.682713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 173/342] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:07.698990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:07.769010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:07.867639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdde9f0-4dd9-40c2-a481-46b56118ef43 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:07.910782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d6af-4de7-4fd8-aa7e-52aae21f6d05 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:07.938649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:07.941265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.943119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.943619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.968221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:07.980981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:07.981917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:07.982419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:07.982913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:07.985518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:07.986382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:07.987168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:07.989558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 170/343] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:07.990571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.006076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.095854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.096436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfee8bf-79cc-4150-97ab-4e6b16f34281 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.120157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123b0b8-b0e3-4da2-a29e-e5c670473c21 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.136912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.162254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.162254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.162254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.162640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.174601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:08.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:08.186000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:08.187326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 171/344] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:08.188025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.224001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 174/345] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:08.225741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ad54-a1fd-4c09-8991-fca618975c34 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.226312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ad54-a1fd-4c09-8991-fca618975c34 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.226793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ad54-a1fd-4c09-8991-fca618975c34 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.228282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 175/346] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:08.229191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ad54-a1fd-4c09-8991-fca618975c34 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.284834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.306490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de469e-78c3-4ae5-9d44-fa0a945a5a11 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.344995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.345563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.345848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.346143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.346874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:08.347182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.349158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6748b2a-2e16-4a9c-abdd-5b47810241b9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.351899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ad54-a1fd-4c09-8991-fca618975c34 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.358331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 172/347] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:08.359653 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.359942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.360223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.382420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:08.383337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:08.415903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943bad-08a3-48fb-9980-613454727057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.416786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.417875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943bad-08a3-48fb-9980-613454727057 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.418403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943bad-08a3-48fb-9980-613454727057 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.418918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943bad-08a3-48fb-9980-613454727057 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.420060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943bad-08a3-48fb-9980-613454727057 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:08.420597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943bad-08a3-48fb-9980-613454727057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.422240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943bad-08a3-48fb-9980-613454727057 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.433967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 176/348] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:08.435108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943bad-08a3-48fb-9980-613454727057 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.459896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 173/349] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:08.460646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.547634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.551861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943bad-08a3-48fb-9980-613454727057 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.572015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8b7ea-6d2c-435a-8c9d-b38dc93e0cab None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.595416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.596685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.597189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.597666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.600110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.601836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.604069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 177/350] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:08.605085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.611232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97815977-474e-4418-909a-29ff4de5568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.612046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815977-474e-4418-909a-29ff4de5568d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.612334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815977-474e-4418-909a-29ff4de5568d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.613781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97815977-474e-4418-909a-29ff4de5568d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.657132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.693213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97815977-474e-4418-909a-29ff4de5568d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.694872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97815977-474e-4418-909a-29ff4de5568d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.719255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1647d4-391c-483a-b139-16e28dde161f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.757987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.758892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.759178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.759459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] PATH_INFO: `/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.760842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe4d5a4173f49d49e0c4ce014da4a98)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:08.762358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:08.764303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 178/351] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/users/dfe4d5a4173f49d49e0c4ce014da4a98 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:08.766124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.807963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.825142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 174/352] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:08.827153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97815977-474e-4418-909a-29ff4de5568d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.883481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dbb93-3121-4bc5-804d-abcafda0e132 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:08.906648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:08.922443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:08.923175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:08.923463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:08.923741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:08.937589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:08.945712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:08.947230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 179/353] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:08.948910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:08.955894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97815977-474e-4418-909a-29ff4de5568d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.002593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.004834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.004834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.004834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.013451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.014935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.017081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 175/354] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.017857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.073178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.087954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c3c2-03ac-46d3-8d2d-6352fd51ae72 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.133601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.135336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.135961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.137312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.138516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] RBAC: Adding query filter params (role.id=969fc1ce10964e99ac25ca5ebc9437a9, user.id=dfe4d5a4173f49d49e0c4ce014da4a98, scope.system=all) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:09.139038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.143777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.147330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fead414-f506-4ae8-a289-1b812be18b40 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.155162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 180/355] 10.222.0.41 () {56 vars in 1308 bytes} [Fri May 27 14:58:08 2022] GET /identity/v3/role_assignments?user.id=dfe4d5a4173f49d49e0c4ce014da4a98&scope.system=all&role.id=969fc1ce10964e99ac25ca5ebc9437a9 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.155854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.193707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.203450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.204786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.205281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.205752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.208475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.210331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f51323-b1a4-498c-b1ff-d5a7efcb4285 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.212679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 176/356] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.214335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52461b6-6b6f-49b5-b9a4-5260aa33837f None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.214814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52461b6-6b6f-49b5-b9a4-5260aa33837f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.215295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52461b6-6b6f-49b5-b9a4-5260aa33837f None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.220885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 177/357] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:09.222846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52461b6-6b6f-49b5-b9a4-5260aa33837f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.303417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52d476-ca3b-4b1e-9603-f55dc288f1e6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.345332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.346015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.346310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.346590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.347358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:09.347673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.349132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.350881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52461b6-6b6f-49b5-b9a4-5260aa33837f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.360031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 181/358] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.360766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.391643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.408526 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.409678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.410125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.410594 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.411585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=22887df109864ec39f82aaf569743a97) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:09.412045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.413872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bfa689-c504-469d-9a37-7dbc99f6acac None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.419900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 178/359] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.421742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef64395-2eb4-4856-9951-f74bee4e1114 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.422322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef64395-2eb4-4856-9951-f74bee4e1114 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.422678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef64395-2eb4-4856-9951-f74bee4e1114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.442723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7ef64395-2eb4-4856-9951-f74bee4e1114 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:09.444431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7ef64395-2eb4-4856-9951-f74bee4e1114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:09.514887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae812058-e0a3-4116-a5aa-69dfaede8a44 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.540406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.558275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 179/360] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:09.560475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90660e8d-f20a-4cab-97f9-5400b695a7da None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.561443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90660e8d-f20a-4cab-97f9-5400b695a7da None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.562694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90660e8d-f20a-4cab-97f9-5400b695a7da None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.564873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 180/361] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:09.565877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90660e8d-f20a-4cab-97f9-5400b695a7da None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.570382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.570382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.570382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.570382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.570744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.570744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648c4cd-0801-4f3f-a585-862ce0149aff None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.577632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 182/362] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.579007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.579414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.579703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.601350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:09.602570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:09.693295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 183/363] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:09.694133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.710449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.722801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90660e8d-f20a-4cab-97f9-5400b695a7da None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.779660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.780548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.781048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.781450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.816776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bf717-74ae-4eef-8280-89592ee32c05 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:09.858184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:09.859750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:09.860411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:09.860738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:09.861017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:09.861859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=22887df109864ec39f82aaf569743a97) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:09.862173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.863619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.870453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 184/364] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=66fc02452abf40a398e393da3430e5d2 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:09.871259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:09.882140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:09.884761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:09.978415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.012077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7acb3-c821-4306-a3af-41679cd1a5bf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.033251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 181/365] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:10.034452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.061347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.062963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.063458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.063859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.077347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:10.087005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:10.088195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:10.089007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 185/366] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:10.089655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.154613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.167657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc5e2e-8ed7-4e2e-880b-1d506832d822 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.217002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.218911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.219451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.220017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.236746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:10.237955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bccf8d8-67b5-421f-b4db-6d29b4c57dc9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:10.245512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:10.246451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:10.247244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:10.250181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 182/367] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:10.251279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.284413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.285348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.285632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.285906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.287439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.289375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.291891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 186/368] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:10.292730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.305721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.395597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939fd8d-253b-4872-9a4d-43690a860640 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.436242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.437495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.437989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.438495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.439445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:10.439971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.441480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ddedc-dc58-4bf2-81a0-23111145c613 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.441846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.451457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 183/369] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:10.452696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.471796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.480495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.481166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.481454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.481736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.482493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:10.482858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.484911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.494077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 187/370] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:10.494842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.567491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88babf00-84d6-463d-98fb-109ce47b64b9 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.567835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.606891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.608106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.608538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5b633-c729-4cfd-9cb3-6509f95d8c52 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.608930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.609473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.611673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.613425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.615712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 184/371] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:10.616785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.647208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.647866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.648149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.648427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.715465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.728740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.729452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.739275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bb31b-0f95-4aa2-9ad1-9eeb888df73a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.776836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.777816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.778769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.779272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.802876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.845826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 188/372] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:10.846783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.859879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:10.861801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:10.946363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:10.957870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953a2a4-2ad6-49af-91eb-2603b0742728 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:10.967383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 185/373] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:10.970245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:10.996267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:10.996856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:10.997140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:10.997414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:10.998240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:10.998551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.000568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.006345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 189/374] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=66fc02452abf40a398e393da3430e5d2 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:11.007141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:11.052529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:11.088921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5564-bb3a-4d95-889b-6938cdbf9ce7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.144411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.147047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.147384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.147667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.150559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.152080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.153942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 186/375] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:10 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:11.160482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:11.169938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7185e4-5ff6-4265-bd5b-8ce5c63ac4b2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.248328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.249259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.250078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.250362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] PATH_INFO: `/v3/projects/1ab0327c3107460fb9252bf46e1b56cc` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.252783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] RBAC: Authorizing `identity:get_project(project_id=1ab0327c3107460fb9252bf46e1b56cc)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.254281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.265013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 190/376] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:11 2022] GET /identity/v3/projects/1ab0327c3107460fb9252bf46e1b56cc => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:11.269874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:11.276413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:11.342214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e2908-74c0-43cd-877d-c78ac6cd0621 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.405204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.406135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.406412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.406690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.407963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.409474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.412388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 187/377] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:11 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:11.413735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:11.417363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:11.417843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba563-e387-4ab2-8c62-f55fa5c0e39c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.476920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.477870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.478154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.478435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] PATH_INFO: `/v3/groups/b6187c83131c4815801ebfcae6fbc7d7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.494623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] RBAC: Authorizing `identity:get_group(group_id=b6187c83131c4815801ebfcae6fbc7d7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.496129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.499370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 191/378] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:11 2022] GET /identity/v3/groups/b6187c83131c4815801ebfcae6fbc7d7 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:11.506313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:11.590686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d08cf-fa3e-48f0-a2a7-4eac4b0f6a81 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.637161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:11.666343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.667370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.667851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.668391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.669518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:11.670031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.672512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ae3e0-6e30-4c76-9037-77d21fd446c8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.685848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 188/379] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:11 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 260 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:11.708878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86716d2-d722-438e-bde0-2fcb9b3c1530 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:11.749132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:11.750455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:11.750991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:11.751483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:11.752548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] RBAC: Adding query filter params (group.id=b6187c83131c4815801ebfcae6fbc7d7, role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=1ab0327c3107460fb9252bf46e1b56cc) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:11.753062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:11.754734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9bd1b-9ebf-4512-9e7b-6bedc0f52ef9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:11.761970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 192/380] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:11 2022] GET /identity/v3/role_assignments?group.id=b6187c83131c4815801ebfcae6fbc7d7&scope.project.id=1ab0327c3107460fb9252bf46e1b56cc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:12.032979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccac54f-8062-44fe-bf11-a3baecb480fc None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.033630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccac54f-8062-44fe-bf11-a3baecb480fc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.034114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccac54f-8062-44fe-bf11-a3baecb480fc None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.035968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 189/381] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:12.041342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.041892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.042397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.065536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:12.067083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:12.098246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:12.158866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 193/382] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:12.193002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccac54f-8062-44fe-bf11-a3baecb480fc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:12.325198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccac54f-8062-44fe-bf11-a3baecb480fc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:12.361926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:12.362743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.363024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.363298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.375368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:12.387079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:12.387970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:12.388813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 190/383] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:12.397573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:12.523142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bed80-d133-4a8b-af7b-0c0157ded60a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:12.567571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:12.568695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.569206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.569686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.570652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:12.571160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:12.572779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:12.582253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 194/384] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:12.587825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:12.672877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:12.741926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79989a3b-82a4-40f7-be00-20180a97251b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:12.786236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:12.787099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.787382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.787658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.788878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:12.790331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e134c-ba3c-42c5-8512-dff86f7567c3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:12.792262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 191/385] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:12 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:12.798510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:12.833606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:12.932237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c48b85-fbb3-44c9-b93c-d5fd43345f65 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:12.936162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a7-e05f-4bdf-aafc-a6643af4f5ba None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.936487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a7-e05f-4bdf-aafc-a6643af4f5ba None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.936757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a7-e05f-4bdf-aafc-a6643af4f5ba None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.938454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 192/386] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:12.947233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.947521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.947803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.977976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:12.978869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:12.979156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:12.979436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:12.981921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:12.983108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:12.985301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:12.986782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:12.988779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 195/387] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:12 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:12.994063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:13.096961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:13.109591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f04d20-a260-4cdc-b9a6-9f1a6691f7c6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:13.123262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 193/388] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:13.155085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:13.165997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:13.166963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:13.167248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:13.167622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/groups/a1d55c7ad236465597fa709698c8d228/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:13.173897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, group_id=a1d55c7ad236465597fa709698c8d228, role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:13.175417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:13.198987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 196/389] 10.222.0.41 () {58 vars in 1442 bytes} [Fri May 27 14:58:12 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/groups/a1d55c7ad236465597fa709698c8d228/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 14:58:13.207609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:13.294688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10db15f-8eaf-4ac6-b5ed-8eed3b2f9315 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:13.341026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:13.341831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:13.342113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:13.342395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:13.366588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:13.376991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:13.378977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:13.382437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 194/390] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:13.383085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:13.552202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090daa9-1eaf-4612-88c2-edb52a2afe00 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:13.595121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:13.595796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:13.596075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:13.596349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:13.597126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:13.597435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:13.598832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:13.613790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 197/391] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:13.619178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:13.737250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:13.740559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7abbfc-e264-4754-b985-3aa53d6dfb23 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:13.777418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:13.778659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:13.779145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:13.779686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:13.781315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:13.782977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:13.785168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 195/392] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:13 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:13.792493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:13.808795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:13.941720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d028cb1-54b5-44c7-a351-dce8d5de645b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:13.991360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:13.992318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:13.992637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:13.992963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:13.994235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:13.995735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda0ce-e4b5-4a0e-bd92-c0709f0325fe None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:13.997906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 198/393] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:13 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:14.004532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.121352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad067-c90e-4aeb-9b50-193aac616802 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:14.158187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:14.159084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.159365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.159642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97/groups/a1d55c7ad236465597fa709698c8d228/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.161619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=22887df109864ec39f82aaf569743a97, group_id=a1d55c7ad236465597fa709698c8d228, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:14.163068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78f114-0996-4588-880f-41af9f9377d3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:14.177790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 196/394] 10.222.0.41 () {58 vars in 1442 bytes} [Fri May 27 14:58:14 2022] PUT /identity/v3/projects/22887df109864ec39f82aaf569743a97/groups/a1d55c7ad236465597fa709698c8d228/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:14.213442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a36f-443e-4f16-a9da-b4debeb74441 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.214065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a36f-443e-4f16-a9da-b4debeb74441 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.214557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585a36f-443e-4f16-a9da-b4debeb74441 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.217885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 199/395] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:14.223991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.224722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.224722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.243399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:14.244793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:14.272173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:14.336251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 197/396] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:14.383339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a36f-443e-4f16-a9da-b4debeb74441 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.504149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:14.535333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585a36f-443e-4f16-a9da-b4debeb74441 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:14.575598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:14.576411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.576727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.577009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.591972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e8886fbe-196d-4813-8c6c-f0e4c1fd3858 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:14.601125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:14.602043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:14.602814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:14.605562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 200/397] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:14.614452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.637276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02961b-7f22-448b-b249-9c4474cdea37 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.637883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02961b-7f22-448b-b249-9c4474cdea37 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.638415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02961b-7f22-448b-b249-9c4474cdea37 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.641835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 201/398] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:14.647607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.647909 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.648182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.689589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:14.690812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:14.732804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be295c3c-9203-4631-890f-75bc2eed3222 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:14.772021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:14.773131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.773614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.774155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.775248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:14.775750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:14.777509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:14.787736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 198/399] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:14.798304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.836908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:14.837879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 202/400] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:14.879762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.911409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cfd84-7384-430c-8d35-d6f0b27dd18b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:14.948061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:14.949286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.949795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.950383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.952037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:14.954743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652d9b5-9736-4d9f-a2b2-879b1cd14b2c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:14.957043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 199/401] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:14 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:14.958588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f440f-7be2-4e54-aab8-3a92a066ec53 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:14.959071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f440f-7be2-4e54-aab8-3a92a066ec53 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:14.959551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f440f-7be2-4e54-aab8-3a92a066ec53 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:14.961477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 200/402] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:14.962462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f440f-7be2-4e54-aab8-3a92a066ec53 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:14.978446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.017302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce063d-535a-4719-b66b-a9e0c9d07618 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.055662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.056655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.056992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.057274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.064792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.069674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e8f17be4-2187-4619-bfb9-1f5f5600c17f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:15.079247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:15.080135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:15.080952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:15.083114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f440f-7be2-4e54-aab8-3a92a066ec53 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.084858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 203/403] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:15.086374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.086668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.086949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.107326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:15.108647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:15.121665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.122805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.123283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.123752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.125359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.127100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.129500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 201/404] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:14 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:15.130519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.193597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 204/405] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:15.194773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.276495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259d23b-ec0c-4bf2-9d65-dbee72dc7257 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.282302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.326449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.327052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.327351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.328405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.329193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:15.329506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.331721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.333948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c90bc3-95c1-4fe6-8a73-21d617c16847 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.347652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 202/406] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:15.349776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.374367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.376100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.377519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.378060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.379093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/groups/a1d55c7ad236465597fa709698c8d228/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.381768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, group_id=a1d55c7ad236465597fa709698c8d228, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.383585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.404917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 205/407] 10.222.0.41 () {58 vars in 1442 bytes} [Fri May 27 14:58:15 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/groups/a1d55c7ad236465597fa709698c8d228/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:15.407210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.551427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eb26e-c5a4-480c-a962-767d30aea8b7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.587035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.588182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.588580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.588950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.601904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:15.612422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:15.613444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:15.614264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 203/408] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:15.619372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.630815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b00a1-208c-4867-b0db-020489106ad2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.650717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.671383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.673082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.673523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.673940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] PATH_INFO: `/v3/groups/b6187c83131c4815801ebfcae6fbc7d7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.675444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] RBAC: Authorizing `identity:get_group(group_id=b6187c83131c4815801ebfcae6fbc7d7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.677088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.679187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 206/409] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/groups/b6187c83131c4815801ebfcae6fbc7d7 => generated 229 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:15.689625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.710940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:15.740085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328e22e-6d7d-4c05-886b-01a084cb0f57 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.779348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.780034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.780317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.780697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.782458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:15.782857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.787407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.793346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 204/410] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:15.801076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:15.893200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee3a07-1ae8-4557-ab92-5f41f6e71a05 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.949827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:15.950692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.950974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.951697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] PATH_INFO: `/v3/projects/1ab0327c3107460fb9252bf46e1b56cc` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.953367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] RBAC: Authorizing `identity:get_project(project_id=1ab0327c3107460fb9252bf46e1b56cc)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:15.955489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4d88f-a570-4ba2-9679-2ff469fff3df None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:15.958288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 207/411] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/projects/1ab0327c3107460fb9252bf46e1b56cc => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:15.959861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6ddb7-36dd-4ca3-bde6-169553e074cb None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:15.960308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6ddb7-36dd-4ca3-bde6-169553e074cb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:15.960621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6ddb7-36dd-4ca3-bde6-169553e074cb None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:15.964144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbda213-9074-49d9-ae6f-de294471cb3a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:15.967233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 208/412] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:15.968609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6ddb7-36dd-4ca3-bde6-169553e074cb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.009259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.010326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.010610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.011181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.012624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:16.014423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c584a3-aee9-4973-b0e8-82deca5d60f3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:16.017195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 205/413] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:15 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:16.018411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.018699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.018977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.044529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:16.053752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:16.056197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:16.059853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:16.108121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6ddb7-36dd-4ca3-bde6-169553e074cb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.148542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.149428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.149773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.150131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] PATH_INFO: `/v3/projects/1ab0327c3107460fb9252bf46e1b56cc/groups/b6187c83131c4815801ebfcae6fbc7d7/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.152338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ab0327c3107460fb9252bf46e1b56cc, group_id=b6187c83131c4815801ebfcae6fbc7d7, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:16.158519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:16.158937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 206/414] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:16.160309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.174090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 209/415] 10.222.0.41 () {58 vars in 1442 bytes} [Fri May 27 14:58:15 2022] PUT /identity/v3/projects/1ab0327c3107460fb9252bf46e1b56cc/groups/b6187c83131c4815801ebfcae6fbc7d7/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:16.204841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.377703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554bd08-7f80-4b60-96fa-51ed7415f6c0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.398859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10769d7-7796-4a52-92a2-62720d6fd538 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.403233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:16.433246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.434287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.434736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.435949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.448489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.449615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.450069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.450496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.451876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:16.452906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:16.453860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:16.455839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 207/416] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:16 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:16.467550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:16.471702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:16.472943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:16.473767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:16.475108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 210/417] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:16.486452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:16.488422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.624572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a14d-aeff-4088-8450-46eb9f675967 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.632949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a242c8-13a8-4eae-897c-5c562bad95e1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.690154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.690730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.691018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.691298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.692107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:16.692369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:16.693791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:16.699596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.700492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.700779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.701054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/groups/a1d55c7ad236465597fa709698c8d228/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:16.706385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] RBAC: Authorizing `identity:create_grant(project_id=0a9a7b7616524d1588d96ce2ea5511b4, group_id=a1d55c7ad236465597fa709698c8d228, role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:16.707823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c1a0d-d782-4458-a7e0-57059ed9d15f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:16.716334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 211/418] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:16.722245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:16.736482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 208/419] 10.222.0.41 () {58 vars in 1442 bytes} [Fri May 27 14:58:16 2022] PUT /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4/groups/a1d55c7ad236465597fa709698c8d228/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:16.792052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:16.871002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb256a-f2eb-4012-ac6c-3e3d4b8221ea None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:16.915904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:16.916606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:16.916939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:16.917219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.009955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605a85-91f0-411e-a22d-5b5c23306fb7 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.010198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605a85-91f0-411e-a22d-5b5c23306fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.010425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64605a85-91f0-411e-a22d-5b5c23306fb7 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.011804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 209/420] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:17.017720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:17.018310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.018601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.018879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.019252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:17.055107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:17.056299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:17.072865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:17.141855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:17.150667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 212/421] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:17.151378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 210/422] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:17.156570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:17.180125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:17.323195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c760206-fe49-4cac-bbc7-e67ae16e63d9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:17.343364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f67b9f-291d-4721-bfa9-20c7ee1f6441 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:17.396491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:17.396491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.396491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.396841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.399660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:17.401178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:17.412927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 213/423] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:17.414661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:17.417917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:17.424467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.424467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.424467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.424885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:17.444601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:17.457038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:17.457946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:17.458710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:17.461485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 211/424] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:17.469427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:17.558247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93d639-836e-4b28-93b8-30e30f0548bc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:17.601010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:17.601765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.607185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.607185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.607185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:17.607185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:17.607185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 214/425] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:17.613982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:17.625505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:17.698889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:17.725247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b098a0-72c9-4145-a9b3-89f146946eca None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:17.739772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecbe1e-035b-4dfb-8a91-a05698ebaab9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:17.777630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:17.778298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.778585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.778864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.779764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=22887df109864ec39f82aaf569743a97) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:17.780085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:17.782899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:17.792965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 215/426] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:17.813392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:17.814108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:17.814394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:17.814669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:17.815464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] RBAC: Adding query filter params (name=member) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:17.815775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:17.819998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe99ca-2e08-4c76-aa6d-87e154a94624 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:17.839547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 212/427] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:17.849834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.033200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ded91d-4c68-4e89-a616-358bbfbda53a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.088822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.089712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.089997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.090275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.100276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:18.210501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:18.211993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:18.219671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259df9a-349a-493a-ad73-4b31bd8fb654 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.219981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259df9a-349a-493a-ad73-4b31bd8fb654 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.220243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259df9a-349a-493a-ad73-4b31bd8fb654 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.221945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 213/428] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:18.227682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.227977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.228259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.250145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:18.252217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:18.345666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 214/429] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:18.354539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 216/430] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:18.361818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.373603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.424505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:18.429334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:18.496157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa8f45-e65f-4b1f-8a67-d67a4275cc15 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.509876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7204c-0066-47df-b7d6-64c468bd81a3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.537253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.538472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.539066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.539536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] PATH_INFO: `/v3/projects/22887df109864ec39f82aaf569743a97` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.541118 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] RBAC: Authorizing `identity:get_project(project_id=22887df109864ec39f82aaf569743a97)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:18.542967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:18.545192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 215/431] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/projects/22887df109864ec39f82aaf569743a97 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:18.550623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.551345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.552397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.552925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.553606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.568119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:18.579057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:18.580021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 14:58:18.580833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:18.582276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 217/432] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:18.592503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.662948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccd465-e50e-4f2a-94fe-ba78cb9093bd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.680774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:18.700193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736805-602a-486d-82da-7de52af5297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.700973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e8786-044b-4ad3-ba60-9ad111f2395f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.701408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736805-602a-486d-82da-7de52af5297e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.701408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736805-602a-486d-82da-7de52af5297e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.701607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736805-602a-486d-82da-7de52af5297e None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.702801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736805-602a-486d-82da-7de52af5297e None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:18.705260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736805-602a-486d-82da-7de52af5297e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:18.705660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:18.711086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 216/433] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:18.712062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736805-602a-486d-82da-7de52af5297e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.749436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.750724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.751214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.751740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.752716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] RBAC: Adding query filter params (name=member) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:18.753232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:18.754855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:18.764850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 218/434] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:18.771134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:18.831775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736805-602a-486d-82da-7de52af5297e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.868608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.869263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.869540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.869812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.870640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=22887df109864ec39f82aaf569743a97) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:18.870946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:18.872366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eccc8-985f-4dcb-8d9f-26334b181c79 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:18.884675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 217/435] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=22887df109864ec39f82aaf569743a97&role.id=66fc02452abf40a398e393da3430e5d2 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:18.898237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b0bc-9eb7-4e7b-804c-c40b8982477a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:18.944064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:18.944678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.944963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.945240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.982258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880f06-55b3-4c97-b8a1-859e2d9c712a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.985525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880f06-55b3-4c97-b8a1-859e2d9c712a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.985525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880f06-55b3-4c97-b8a1-859e2d9c712a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:18.985525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 218/436] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:18.996474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:18.996474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:18.996474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.020658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:19.021187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.024471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:19.111131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.114514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.126089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.175203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 219/437] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:19.201309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.269775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 219/438] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:58:19.275435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.355758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.361114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b759f58-f5b4-4d97-adbf-1d0e8ac07022 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.425244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.426376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.426658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.426932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.437634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a64d4-525f-4597-9550-7044514fd6dc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.454470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:19.461030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:19.464027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:19.464852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 220/439] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:19.468042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.497037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.498354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.498636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.499188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.500907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.505685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.505685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 220/440] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:19.509564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.523796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.623264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a97f-8168-4d2a-a374-f4e390d77311 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.643295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd4b37-0eea-40bd-b45f-ec32bfce02a3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.674977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.681922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.682591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.682876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.683153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.683908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:19.684222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.685760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.686597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.688266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.688809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.689205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.690836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.692283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.695501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 221/441] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:19.696925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 221/442] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:19.700916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.702533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:19.809238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.809600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b87ad-a191-41ee-a91f-fe83a994d507 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.813153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a6a65-0fa2-41e2-9e87-f8715ede535e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:19.844101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.844796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.845085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.845364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.846191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=66fc02452abf40a398e393da3430e5d2, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:19.846509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.847469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:19.847960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e102d-6e32-4abc-8aa2-dd3691b70718 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.848119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.848403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.848711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.853836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 222/443] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=66fc02452abf40a398e393da3430e5d2 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:19.855039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8a936-231e-403f-9750-c2f34d3755fa None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.855328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8a936-231e-403f-9750-c2f34d3755fa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.855607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8a936-231e-403f-9750-c2f34d3755fa None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.857045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 223/444] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:19.863652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:19.863950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:19.864227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:19.883547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:19.884838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:19.939739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:19.941255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:19.949809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:19.967389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 224/445] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:19.992403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.058601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.063284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 222/446] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:20.068590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.117955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41505e96-5f1a-437d-a5c4-6dfb1e4d837d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.157979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.158793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.159080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.159356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.173087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:20.181458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:20.182340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:20.183126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 225/447] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:20.188625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.197795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc9e00-9460-4093-9142-161f327d42b1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.236201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.237334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.237805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.238274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] PATH_INFO: `/v3/projects/1ab0327c3107460fb9252bf46e1b56cc` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.239922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] RBAC: Authorizing `identity:get_project(project_id=1ab0327c3107460fb9252bf46e1b56cc)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.241131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.243303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 223/448] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/projects/1ab0327c3107460fb9252bf46e1b56cc => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:20.248775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.250446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.304865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03fb18-3e11-48ec-9429-75c52676d1e9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.317617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.339507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.340167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.340474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.340759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.341506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:20.341813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.343197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.351985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 226/449] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:20.357404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.357850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee7720-3dae-427f-89c5-450c49e6ed3f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.395366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.396232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.396548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.396827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] PATH_INFO: `/v3/groups/b6187c83131c4815801ebfcae6fbc7d7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.398001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] RBAC: Authorizing `identity:get_group(group_id=b6187c83131c4815801ebfcae6fbc7d7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.399456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.405719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 224/450] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/groups/b6187c83131c4815801ebfcae6fbc7d7 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:20.407732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.458698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24284f-bec6-42b2-ba77-750ad4493897 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.470038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.494992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.495636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.495920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.496200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.507585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4a7f-b906-4e32-8241-68ba2d09aebb None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.542251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.542898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.543179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.543454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.544271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] RBAC: Adding query filter params (group.id=b6187c83131c4815801ebfcae6fbc7d7, role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=1ab0327c3107460fb9252bf46e1b56cc) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:20.544617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.546050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.549490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.553264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 225/451] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/role_assignments?group.id=b6187c83131c4815801ebfcae6fbc7d7&scope.project.id=1ab0327c3107460fb9252bf46e1b56cc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:20.568533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.569965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.676654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 227/452] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:58:20.682365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.788514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a1bd-8d17-43b0-b7ed-83623bcc60f0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.821398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.822217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.822492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.822771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] PATH_INFO: `/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.823941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a9a7b7616524d1588d96ce2ea5511b4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.825432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.827116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 226/453] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/projects/0a9a7b7616524d1588d96ce2ea5511b4 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:20.833435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:20.845345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:20.929695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a2dd-0882-49df-b85b-81a368e393fd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:20.962943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:20.964021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:20.964299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:20.964602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] PATH_INFO: `/v3/groups/a1d55c7ad236465597fa709698c8d228` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:20.965746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] RBAC: Authorizing `identity:get_group(group_id=a1d55c7ad236465597fa709698c8d228)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:20.967178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fded0d-730f-43a4-b001-42a75cb96e33 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:20.968852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 228/454] 10.222.0.41 () {56 vars in 1180 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/groups/a1d55c7ad236465597fa709698c8d228 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:20.974501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:21.039719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:21.073034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b08aef-51ee-462f-9f40-abb259ca54d5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:21.106609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:21.107263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:21.107550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:21.107835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:21.108692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] RBAC: Adding query filter params (group.id=a1d55c7ad236465597fa709698c8d228, role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:21.109012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:21.111220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4498e-e011-4c7c-bd51-30cb0d564704 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:21.117134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 227/455] 10.222.0.41 () {56 vars in 1376 bytes} [Fri May 27 14:58:20 2022] GET /identity/v3/role_assignments?group.id=a1d55c7ad236465597fa709698c8d228&scope.project.id=0a9a7b7616524d1588d96ce2ea5511b4&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:23.974985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a57915-fae2-42f3-b270-c55bac262569 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:23.975500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a57915-fae2-42f3-b270-c55bac262569 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:23.975605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a57915-fae2-42f3-b270-c55bac262569 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:23.977221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 229/456] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:23.982468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1267-04a2-4310-b9ba-5c4b96644161 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:23.982961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1267-04a2-4310-b9ba-5c4b96644161 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:23.983037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c1267-04a2-4310-b9ba-5c4b96644161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.009706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7b0c1267-04a2-4310-b9ba-5c4b96644161 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:24.011297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7b0c1267-04a2-4310-b9ba-5c4b96644161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:24.095167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 228/457] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:24.141832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a57915-fae2-42f3-b270-c55bac262569 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.238126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.248909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b97a1-df60-4960-90ac-658dfb115bb2 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.249505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b97a1-df60-4960-90ac-658dfb115bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.250582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b97a1-df60-4960-90ac-658dfb115bb2 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.252207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 229/458] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:24.257835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba80e4-53b0-4d58-ae44-4f0ee9742a29 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.258312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba80e4-53b0-4d58-ae44-4f0ee9742a29 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.258739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba80e4-53b0-4d58-ae44-4f0ee9742a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.267271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a57915-fae2-42f3-b270-c55bac262569 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.293553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1bba80e4-53b0-4d58-ae44-4f0ee9742a29 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:24.295032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1bba80e4-53b0-4d58-ae44-4f0ee9742a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:24.307144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.308155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.308610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.309054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.321964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.324003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-16e6f2e5-de5b-4109-b8e4-3b329f5b760c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:24.347319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:24.348320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:24.349182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:24.349182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:24.349182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.349182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:24.351782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 230/459] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:24.353419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba747e1-8f46-4a9e-9aa6-dab352093f8a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.353868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba747e1-8f46-4a9e-9aa6-dab352093f8a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.354302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba747e1-8f46-4a9e-9aa6-dab352093f8a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.356178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 231/460] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:24.357076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba747e1-8f46-4a9e-9aa6-dab352093f8a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.399694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.421287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 230/461] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:24.424055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.424430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.424810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.455434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:24.456655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:24.503031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba747e1-8f46-4a9e-9aa6-dab352093f8a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.527173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 231/462] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:24.532589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.539216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.552914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.553997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.554489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.554993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.555970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:24.556542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.558406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:24.582546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 232/463] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:24.588964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.643884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10788e-acb6-44fc-a8ff-bb07bf28e618 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.669450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.683195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.683988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.684271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.684592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.696804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.698373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:24.704403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63959785-3333-4ce9-8d05-1f8fb03aa444 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:24.718170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:24.719116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:24.720107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:24.720107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:24.720107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.720107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:24.720107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 232/464] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:24.725677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.750616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.751845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.752332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.752873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.753860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.755851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:24.777700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 233/465] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:24 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:24.778625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.851016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad268b-7710-4ead-b827-19a0931dd84b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.860729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.891206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.892190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.892598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.892941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.904990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.907170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-a5a4a96b-40b7-4b7a-9707-721dcd487c16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:24.926937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:24.927859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:24.928940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:24.928940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:24.928940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:24.928940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:24.928940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 233/466] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:24.930342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1b86-8cdd-4d7a-92d0-73b036aef7d7 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.930750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1b86-8cdd-4d7a-92d0-73b036aef7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.931100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a1b86-8cdd-4d7a-92d0-73b036aef7d7 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.932485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 234/467] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:24.937355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a1b86-8cdd-4d7a-92d0-73b036aef7d7 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:24.943920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0f532-e94e-440c-9e38-d6210675ccd3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:24.970076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:24.993950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:24.994695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:24.994901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:24.995186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:24.995910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:24.996225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:24.998732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93920851-6157-46f9-86a3-d2f488887e6b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.014192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 234/468] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:25.015472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.016304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.016642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.037298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:25.038921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:25.075179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a1b86-8cdd-4d7a-92d0-73b036aef7d7 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.134721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.135825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.136370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.136750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.137601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:25.138036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.139537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.139917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 235/469] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:25.141328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.157022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 235/470] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:25.163029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.193994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:25.272481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:25.290955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853fc15-509e-498c-983d-86ae2bbe421c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.292171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49da6-f144-40f7-a2ac-36e3ed91bb7d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.327095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.327766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.328044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.328545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.329571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.331014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.331674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.332339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.332655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.332933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.333638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.335072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b51e25-cf61-4241-8303-59d5187f7a24 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.352593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 236/471] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:58:25.353750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.363299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 236/472] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/users => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:25.365935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88cf5-f0ca-4859-99d9-cc75d100dfb2 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.366491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88cf5-f0ca-4859-99d9-cc75d100dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.366971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c88cf5-f0ca-4859-99d9-cc75d100dfb2 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.369784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 237/473] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:25.377549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.377880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.378110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.398984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:25.400273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:25.482247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 238/474] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:25.525350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a705-9fb1-4817-b5a2-bf02bdf60cd4 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.534348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.597408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:25.597859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.599327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.599920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.600397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.617888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.619761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.622346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:25.649460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:25.650363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:25.651221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:25.651221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:25.651221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:25.651221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:25.651221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 237/475] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:25.657935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.710849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5715e22-8130-4668-a5d7-a6b7a2dc953e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.755412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.756097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.756385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.756690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.757448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.758865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.766618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.sql.core [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 14:58:25.766618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 14:58:25.766618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [parameters: {'id': '51bd6c6759c74ff399249071555915da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 14:58:25.766618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() May 27 14:58:25.783622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return ref.to_dict() May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:58:25.784534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application yield resource May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.rollback() May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.force_reraise() May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise self.value May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application session.commit() May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 14:58:25.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._prepare_impl() May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.flush() May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._flush(objects) May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application compat.raise_( May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application flush_context.execute() May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rec.execute(self) May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 14:58:25.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:58:25.787192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:58:25.788041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [parameters: {'id': '51bd6c6759c74ff399249071555915da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:25.789843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 14:58:25.791314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:25.792257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 239/476] 10.222.0.41 () {60 vars in 1131 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:25.792257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.800130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e914261-cb26-46dd-837e-2a618a0d2df3 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.846954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:25.848055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:25.848341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:25.848923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:25.849666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:25.849976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:25.852177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:25.872860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 238/477] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:25.874911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:25.909533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:25.929295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc45e2b-347c-4c37-8302-7e2fd38686c1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:25.950514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:26.003432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e3329-d6ad-4466-8f27-6fd94d2d77c1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:26.006909 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:26.007371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:26.007619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:26.007901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:26.029940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:26.049620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:26.050618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:26.051096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:26.051601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:26.052563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:26.054197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b17a-c186-4d2c-8554-68dea60671c7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:26.055734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:26.056708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 14:58:26.057532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:26.061502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 240/478] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:58:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:26.073095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:26.095534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 239/479] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:25 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:58:26.223393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c07c3-7510-4c9c-b808-85fc3466c39f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:26.284213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:26.284999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:26.285290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:26.285570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:26.286348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:26.286667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:26.288104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08113b5-d45c-461f-bd01-c1944923bbcf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:26.299163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 241/480] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:27.406297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691047-a2b8-4d51-a0b6-9fc50a9dc853 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:27.406869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691047-a2b8-4d51-a0b6-9fc50a9dc853 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:27.406869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691047-a2b8-4d51-a0b6-9fc50a9dc853 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:27.408509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 240/481] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:27.413880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:27.414327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:27.414503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:27.442532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:27.443730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:27.446050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:27.546125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 242/482] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:27.581903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691047-a2b8-4d51-a0b6-9fc50a9dc853 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:27.714501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691047-a2b8-4d51-a0b6-9fc50a9dc853 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:27.726707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:27.753568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:27.754724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:27.755208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:27.755684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:27.766695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:27.774610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:27.775508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:27.776327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:27.779029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 241/483] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:27.785001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:27.955848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7ca77-074e-445d-bc0f-247746511d69 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.003829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.004432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.004738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.005014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.005770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:28.006078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:28.007464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51720607-50c0-47de-981d-c4be7fe12f57 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:28.016949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:28.017356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 243/484] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:28.026168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.136528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430634-5e25-4373-9a4e-4970e7989e11 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.136528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430634-5e25-4373-9a4e-4970e7989e11 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.136528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23430634-5e25-4373-9a4e-4970e7989e11 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.141251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 244/485] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:28.142365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.142756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.143154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.185814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:28.186933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ad3b5-318e-4c93-a453-52533ad9a45f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.187403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:28.267199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 245/486] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:28.274476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.275453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.275890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.276313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.288372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:28.290194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:28.308026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:28.315694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:28.316755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:28.317814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:28.317814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:28.317814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:28.317814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:28.317814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 242/487] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:28.322263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.382066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:28.447076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d331d07-ad3b-4867-94ad-d16d4ac3c60d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.483944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:28.488805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.490015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.490571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.491073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.497020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd0b46-92fa-492a-bde3-f336b080269d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.504587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-82f171fb-a98b-4512-bd56-8df3eabd8d15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:28.513144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:28.514044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:28.514912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:28.518792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 246/488] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:28.520309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ee47-3534-457e-93df-9b1ca9418c1d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.520793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ee47-3534-457e-93df-9b1ca9418c1d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.521248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298ee47-3534-457e-93df-9b1ca9418c1d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.522918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 247/489] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:28.524288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ee47-3534-457e-93df-9b1ca9418c1d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.543727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.544769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.545121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.545471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.546740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:28.547589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:28.549462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f20d75-21f2-4920-89c0-ba25c3b019d0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:28.567035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 243/490] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:28.568439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.568825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.569429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.591808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:28.593401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:28.649542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298ee47-3534-457e-93df-9b1ca9418c1d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.692467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 244/491] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:28.692467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.701885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.702565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.702848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.703166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.704013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:28.704375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:28.706853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:28.720685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 248/492] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:28.722344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.727287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:28.782565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:28.831784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b55a5d-e8b1-4ac6-92a4-c48dc6540da5 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.834114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966644aa-2a1d-4dad-99f8-e1cd7367f74d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:28.870071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.870868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.871196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.871477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.872041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:28.873113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:28.873760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:28.874348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:28.883345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:28.883871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:28.885484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:28.892936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:28.893910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:28.894696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:28.899232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 249/493] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:28.900312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:28.909215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:28.910204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:28.911097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:28.913923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 245/494] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:28.915085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:28.994037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.013874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a9c0-f3d8-49f2-a06a-60af8dc184f8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.029994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.035452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd2118-9d6e-4542-9a77-b1269620d197 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.054926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.055722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.056002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.056274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.068296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.069818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.079151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.080127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.080668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.081138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.082097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:29.082614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.084218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93b5c4-6d5c-4dd2-9360-962a836d01eb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:29.089102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:29.089936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:29.090757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:29.090757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:29.090757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.090757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:29.090757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 250/495] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:29.091530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.097468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 246/496] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:29.099025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391cf9-260f-4ab3-8431-03e0a1689500 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.099537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391cf9-260f-4ab3-8431-03e0a1689500 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.100030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391cf9-260f-4ab3-8431-03e0a1689500 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.105851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 247/497] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:29.105998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391cf9-260f-4ab3-8431-03e0a1689500 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.208060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1993d-c347-42d6-b700-41c2665bf277 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.213772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391cf9-260f-4ab3-8431-03e0a1689500 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.242813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.243479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.243763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.244039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.244900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:29.245209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.246663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd971836-8d3a-4e17-a0c8-bc74f185f6a3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.247397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.248057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.248490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.248681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.249360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:29.249672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.251145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.252675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.265207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 248/498] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:29.266120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.283779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 251/499] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:29.285086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ab6b-da8a-418f-bb17-1aaad5bff48e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.285383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ab6b-da8a-418f-bb17-1aaad5bff48e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.285732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ab6b-da8a-418f-bb17-1aaad5bff48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.303028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8fc7ab6b-da8a-418f-bb17-1aaad5bff48e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:29.304245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8fc7ab6b-da8a-418f-bb17-1aaad5bff48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:29.321259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.369185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cdec7-4986-42d4-993c-7ef8396e49de None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.372835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 252/500] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:29.376419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70454-8199-4862-b3fd-d5c7b61bd976 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.376419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70454-8199-4862-b3fd-d5c7b61bd976 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.376419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a70454-8199-4862-b3fd-d5c7b61bd976 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.377087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 253/501] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:29.377795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a70454-8199-4862-b3fd-d5c7b61bd976 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.408643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.409435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.409716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.409987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.419898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.421377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:29.439534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:29.440408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:29.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:29.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:29.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:29.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 249/502] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:29.441580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.489404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.491226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a70454-8199-4862-b3fd-d5c7b61bd976 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.525304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.526087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.526365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.526634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.533902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.535363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.544358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df0d2a-3eb8-428f-891b-352dd77bc3cb None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5a7565a8-a16d-4d83-9be6-01c7b57d4eb6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:29.551721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:29.552810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:29.553733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 254/503] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:29.554646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.554908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.555180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.572260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:29.573438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:29.580989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.582340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.582983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.583314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.583598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.641599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 255/504] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:29.642684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.665576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.667033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.712032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.742997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee8b77-64c6-410d-a440-e51450138fac None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.765643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 250/505] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:29.767044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.778541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.779332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.779618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.779902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.789945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.791420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.793157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:29.809676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:29.810529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:29.811356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:29.811356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:29.811356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:29.811356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:29.813083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 256/506] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:29.814058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.869354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d3f8e-5c9a-4e86-9467-55fa19f62c46 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.903817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.904525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.904811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.905083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.905795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:29.906097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.907927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.916773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0377cf3-4c6e-44ca-be59-70866342117e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:29.919229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 251/507] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:29.920019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:29.929039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:29.951973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:29.952675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:29.952957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:29.953234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:29.954863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:29.955178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:29.956659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:29.975200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 257/508] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:29.975996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.024309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea744-f71e-4210-a222-78db91dd5eeb None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.027529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.069350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.070435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.070857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.071269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.074153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196fe71-a3a8-4cb7-87a1-e808c1456460 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.084174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:30.095808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:30.096738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:30.097570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 252/509] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:30.098755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.110939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.111777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.112065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.112339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.122639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.124104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:30.142026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:30.143045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 258/510] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:30.143950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.171582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.207670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8a0de-a214-440d-b8b6-ddd56cf1ed90 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.241516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.242185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.242469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.242621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19f60b-3c73-4d86-b815-55b735b42b5e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.242743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.243460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:30.243777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.244908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.245208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.257984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 253/511] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:30.258801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.278713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.279510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.279797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.280074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.287406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.288895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:30.303770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:30.304665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:30.305546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 259/512] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:30.306082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.361911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b0ef-9794-42aa-a167-80d8d31d5f41 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.393489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.403300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.403951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.404237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.404559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.418544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed937660-d48d-4588-a813-24f5c33119bc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.453163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.453864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.454156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.454437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.455188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:30.455503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.456954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.465517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 260/513] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:30.466334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.478332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.479848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.496311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.568057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eaf88-7d4b-4827-8e2a-93a645960fd4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.579203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 254/514] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:30.579917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.602156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.602823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.603109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.603397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.604114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:30.604426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.605893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.607160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.619447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 261/515] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:30.620205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.681872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce97fd-835f-41be-abd8-88c58c79d539 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.706572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.716090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.716781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.717060 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.717333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.718033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.720309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.722833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3caab-1c1b-40c8-bd7e-44a042d6b710 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.752483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 255/516] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:30 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:30.752483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.772149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.773155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.773430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.774120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.775021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:30.775409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.777678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.809460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 262/517] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:30.809460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.848842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:30.870702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a057f2-72a7-4598-8ad9-fa71bcdc198a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.909024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.909839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.910122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.910404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.923458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.925380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.930047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e045088-be77-4896-81b2-d05cecad47b0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:30.945013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:30.946090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:30.947044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:30.947044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:30.947044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:30.947044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:30.947044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 256/518] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:30.947332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:30.977223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:30.978026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:30.978309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:30.978585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:30.991846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:30.993377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:30.996659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:31.014002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:31.014894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:31.015754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:31.015754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:31.015754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:31.015754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:31.015754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 263/519] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:31.017143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.056281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ee638-dd9e-498e-98cd-add5e4e7cc71 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.092036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.092869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.093159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.093436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.095419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.101009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.103007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:31.122210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:31.123250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:31.124275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:31.128135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 257/520] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:31.129258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.136036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017808a0-1261-4296-9367-6f4b44081e80 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.178397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.179055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.179344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.179625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.240560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fda96-e078-44a3-b2e0-a61fce0b5010 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.249659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.257669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.259394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.277764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.278429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.278713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.279131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.279713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:31.280025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.281913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.295414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 258/521] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:31.297861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.357922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.372899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 264/522] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:31.372899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.415713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98d52c-df77-4457-a32f-724cdd9d7d5b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.452929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.453943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.453943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.454372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.455103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:31.455413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.456914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.470609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 259/523] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:31.471694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.478742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.487166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83398d6-8768-4303-bb53-1f3d27aeff65 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.525117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.525898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.526180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.526457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.527294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:31.527656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.529338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.554468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 265/524] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:31.554468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.579895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310e759-ffb4-4c09-8711-27d08d938353 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.594286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.616704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.617500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.617777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.618048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.626022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.627557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:31.642812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:31.643809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:31.644741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 260/525] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:31.645929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.663545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362fca-34da-47be-a302-4e29a4ee9bca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.700165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.701152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.701428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.701704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.712437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.714165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:31.732064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:31.733262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 266/526] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:31.735111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.748516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.760315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86529-2910-4aaf-96c2-3e3d3a558f11 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.798281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.799648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.799932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.800211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.809087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.810832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:31.827152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:31.828028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:31.829426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:31.833688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 261/527] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:31.834479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.843437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba465e-9e97-4fb2-85a6-c7d1bb6ab449 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.873244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:31.885508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:31.886127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:31.886412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:31.886694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:31.887531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=36672cdeb2f148698b116e06095955d2) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:31.887856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:31.889340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:31.895803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 267/528] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/role_assignments?user.id=36672cdeb2f148698b116e06095955d2&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:31.896526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:31.980608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5083aa-d34c-4c62-a203-9ef09bdcb7a5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:31.991420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.030245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.030912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.031202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.031582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.032884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.033279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.034892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.036832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a28b3b-07c5-4a9e-b26a-ad4abfe105dc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.079691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 262/529] 10.222.0.41 () {56 vars in 1139 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.080604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.082999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.084122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.085558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.086023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.086921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.087380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.089217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.107217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 268/530] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.108350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.194069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.205561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41259d0-6237-4147-b5f2-aaa7f779f7cd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.242713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.243620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.244118 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.244600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.245573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.246063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.247692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.257780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ff75-d927-4244-a614-46ea61c302ca None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.269277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 263/531] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.270445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.289927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.296086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.296784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.297078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.297413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.372481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.379321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f97c0-23a5-4441-a156-b0f4699e3c82 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.384702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.417186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.418372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.418858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.419336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.427518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.430876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:32.446112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:32.446970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:32.447803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:32.449939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 264/532] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:32.451083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.467604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.496795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 269/533] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:32.497962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.532896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.565798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b910af2-180d-47b2-878b-4ec5729b1993 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.602295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.603728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.604242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.604758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.605791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=c1a532f2e8b446b7a03b38b3dda377a9) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.606322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.608062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.608891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee7db6-2eba-4e99-aa26-16bdc0ad9a5b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.615069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 265/534] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/role_assignments?user.id=c1a532f2e8b446b7a03b38b3dda377a9&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.616204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.649745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.649950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.650239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.650519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.651379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.651779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.653308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.668643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 270/535] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.684475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:32.768965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.819673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7325c-09f2-4733-9645-10fb5892ce56 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.868915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:32.876689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d749c-1612-4308-9c68-c722a6fe3832 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:32.893375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.894520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.895031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.895640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.914913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:32.915585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:32.915874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:32.916157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:32.920479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:32.920479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.920479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.922363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:32.924308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:32.932755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 271/536] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7dffd8da-40ef-401b-90d8-71f394354ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:32.965367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:32.966276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:32.967121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:32.967121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:32.967121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:32.967121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:32.969618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 266/537] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:32.976485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.145825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f501d-c21c-4ae5-8e54-3f1b47b65bc4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.151720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d22e2-6f1a-4970-8bd3-32ecd87cfef6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.152362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d22e2-6f1a-4970-8bd3-32ecd87cfef6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.152913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d22e2-6f1a-4970-8bd3-32ecd87cfef6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.154930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 267/538] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:33.162268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.162926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.163434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.183490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:33.185290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:33.206649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.207350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.208145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.208459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.209212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:33.209533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:33.211512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:33.231786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 272/539] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:33.236604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.257886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:33.282908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 268/540] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:33.307698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.351865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fcb21-418f-4795-84b2-87bc064c7c08 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.421302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.422044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.422448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.422739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.424266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:33.441601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:33.443101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:33.475079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:33.475983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:33.476875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:33.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d059-95eb-4f6b-bc1e-bb7cc6ccfec7 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.489392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 273/541] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:33.491239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.549961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.551263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.552562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.553662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.577620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:33.585896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:33.586760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:33.587571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 269/542] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:33.593393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.654974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:33.683017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8a56c-122a-4ddc-b0d4-f1ed0f49271f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.718743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.719984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.720282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.720595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.721863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:33.722325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:33.723965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:33.727391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e59658-cb8b-46f7-9ec6-4652080909f7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.746000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 274/543] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:33.749433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.756919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:33.792129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.793259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.793738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.794205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.795144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:33.795644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:33.797365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:33.806271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 270/544] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:33.816480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:33.944205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b3613-6017-429e-84ef-032585e1dca9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:33.993524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:33.994251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:33.994536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:33.994814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:33.995657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=9f3ecb0e259e4ed89f354e0e95927f10) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:33.995970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:33.997820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:34.003030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:34.005855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 275/545] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/role_assignments?user.id=9f3ecb0e259e4ed89f354e0e95927f10&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:34.051855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6906d9-1959-4999-b793-921ff1463c7a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:34.098073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:34.098901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.099185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.099460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.114982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:34.116516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:34.141379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:34.142381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:34.143216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:34.143216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:34.143216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:34.143216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:34.143216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 271/546] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:34.153646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:34.305233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bac0e-2b76-45cf-8880-a960187bb7ba None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:34.371294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:34.372043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.372334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.372640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.375845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:34.375845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:34.375845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3643aa-eeb7-4ec8-8f94-f498c68d35eb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:34.396470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 276/547] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:34.398324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:34.473603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:34.563515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730d62-9626-41d6-accf-8191d801dfb9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.564440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730d62-9626-41d6-accf-8191d801dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.564998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730d62-9626-41d6-accf-8191d801dfb9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.566887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 277/548] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:34.572123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.572416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.572726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.572944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ce9fc-0b1c-4128-80af-ba20943ba299 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:34.608926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:34.610858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:34.616464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:34.617341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.617628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.617908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.637895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:34.639380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:34.673640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:34.675105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:34.676201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 272/549] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:34.681127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:34.696574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:34.712108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 278/550] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:34.761061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:34.848531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bbf23-f922-4e39-9057-8a1d893278ef None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:34.856660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:34.870975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffafea7-95e4-4689-912c-32f697988a49 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:34.908969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:34.909653 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.909938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.910218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.911059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:34.911369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:34.915010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:34.915821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:34.916104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:34.916380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:34.917476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:34.938356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:34.946189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:34.947337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:34.948656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 279/551] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:34.960663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 273/552] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:34.961502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:34.966304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.068926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.159538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e575f-b520-40f5-8475-bb4b4752d2f9 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.176801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d50d0-cc3b-40af-9da2-8de8bbe26fcf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.228097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.228805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.229094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.229382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.262044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.262724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.263008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.263286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.264040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:35.264348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:35.265766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:35.280853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.288631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 280/553] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:35.302044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.370691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:35.372171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa253e-1a60-4525-857e-d1dc8945db9d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:35.446215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dced14-e8d4-4fc7-969c-0d24e5962922 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.460245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.488809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.489631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.489915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.490192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.505281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:35.507203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:35.524398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 274/554] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:58:35.529627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f8f4-6b83-4109-90e6-57b351ecca05 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.529912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f8f4-6b83-4109-90e6-57b351ecca05 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.530189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66f8f4-6b83-4109-90e6-57b351ecca05 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.531448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 275/555] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-67bae57c-5263-41f6-a475-3b4df21d265d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:35.536563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:35.537641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:35.539298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:35.539298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:35.539298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:35.539298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:35.539298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 281/556] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:35.539925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66f8f4-6b83-4109-90e6-57b351ecca05 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.540069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.540069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.540069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.560854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:35.562020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:35.609520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.640519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 282/557] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:35.642312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.679563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b66f8f4-6b83-4109-90e6-57b351ecca05 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.718310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.719573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.719934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.720435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.735457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:35.737425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:35.753583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:35.756072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:35.757819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 276/558] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:35.759204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.769293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663ace7-bf0e-4976-9560-d8210bc07c13 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.813263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.814730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.815296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.815746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.816820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:35.817565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:35.819138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882dd12e-1870-4730-b276-703a50832e19 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:35.835026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 283/559] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:35.836737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de5c3-2c09-4645-ac26-195ba9446d1f None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.837203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de5c3-2c09-4645-ac26-195ba9446d1f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.838414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440de5c3-2c09-4645-ac26-195ba9446d1f None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.840371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 284/560] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:35.841277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440de5c3-2c09-4645-ac26-195ba9446d1f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.873938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe852d1d-6e94-4ed6-9f7b-e20d94317786 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:35.875989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:35.912341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:35.913196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:35.913500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:35.913756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:35.924310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:35.932076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:35.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:35.933976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:35.936898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 277/561] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:35.939097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:35.959345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440de5c3-2c09-4645-ac26-195ba9446d1f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.002159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.003056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.003609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.003891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.004642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.004954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.006492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32c693-25a6-48f7-afef-5ab6eea7fa05 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.019827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 285/562] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.021213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.021503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.021781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.040126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:36.041350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:36.048461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8de081-b544-437e-beb3-994a29f677c8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.053032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.086394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.087195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.087472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.087752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.096760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.098223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:36.117381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:36.118380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:36.119320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:36.126002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 278/563] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:36.126002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.126225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 286/564] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:36.126225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.163176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.232964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c7e16-fe8b-4280-bf82-76de1fa7127a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.244666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9079-0022-48c1-a8d5-dd3e655cf487 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.269347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.270201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.270485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.270766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.278940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.280437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.282925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.283728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.284057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.284299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.285119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.285459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.286915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.296082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 279/565] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.296984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:36.297942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:36.298880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:36.299813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 287/566] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:36.300313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.307741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.397668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.415030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12b04-a200-4da1-a531-e1c1f48ffbd2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.419377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1df16-7c71-46a8-a697-17e92f9ec1f7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.452935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.454132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.454415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.454803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.455280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.455632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.455947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.456108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.456353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.456685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.457604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.467413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:36.476029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:36.477023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:36.477854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:36.478478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 280/567] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.478478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.480728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 288/568] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:36.481406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.544161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.583663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e1fc4-d251-4d81-94fb-2feeb23021b5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.590625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f4443-a2cc-4b23-8478-355553cec5a0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.618342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.618992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.619276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.619554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.620339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.620684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.622127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a9ea6-9a8c-49d1-a542-b4d44c5fdca1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.625316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.626227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.626512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.626790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.634186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 289/569] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.635602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7975c9-f802-4c67-aa69-e79d2cd249c6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.635890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7975c9-f802-4c67-aa69-e79d2cd249c6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.636166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7975c9-f802-4c67-aa69-e79d2cd249c6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.637418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 290/570] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:36.638100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7975c9-f802-4c67-aa69-e79d2cd249c6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.641859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.643349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.645259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:36.662451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:36.663448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:36.664326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:36.664326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:36.664326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:36.664326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:36.664326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 281/571] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:36.664627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.742375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7975c9-f802-4c67-aa69-e79d2cd249c6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.770341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d6e42-a6d7-459e-ba5f-ba05d3495b65 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.771949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.784744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.785411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.785691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.785965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.786704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.787009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.788387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.796409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 291/572] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.801614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.804351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.805190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.805476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.805758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.816394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.817904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-75ef1e3f-9e58-4a68-a509-f33db066d20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:36.835724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:36.836660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:36.837516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:36.837516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:36.837516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:36.837516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:36.837516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 282/573] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:36.838422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.838719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.839264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.856311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:36.857531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:36.900226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:36.905725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c63ac-9a73-4b8d-a19c-4fd82fc20159 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:36.927461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 283/574] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:36.928293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:36.940086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:36.940914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:36.941130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:36.941409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:36.942229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=5b7e595d067a42dca56f0ef5fe8485c1) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:36.942539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:36.943995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:36.949696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 292/575] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/role_assignments?user.id=5b7e595d067a42dca56f0ef5fe8485c1&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:36.950480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.009190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.058845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50dc5a-1627-46cd-9694-8600361e8d34 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.072163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7bf3b-2993-4a69-bdf6-c6a66c19c1c9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.100161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.101204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.101483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.101803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.102551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.103139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.104480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.112746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.113560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.113846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.114129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.117643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 284/576] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.118722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.127227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.128750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:37.147793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:37.149064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 293/577] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:37.149977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.168469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.241430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c383562-28bb-4678-9a64-4241fc445032 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.268025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e45a4-cdef-4910-b961-73445bc7ee5b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.269892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.280365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.281085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.281373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.282193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.282924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.283238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.284729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.297902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 285/578] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.298609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.309312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.310070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.310413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.310697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.322109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:37.330129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:37.331058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:37.331870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 294/579] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:37.333207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.411976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.418483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc17b5-96e1-4079-914b-e313b7877cf6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.444189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27f5f-8eac-4fe5-bea9-6e3d7326ed66 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.458473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.459276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.459562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.459841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.467432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.468933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.481114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.482008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.482487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.482955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.483877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:37.484158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:37.485817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 286/580] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:37.486872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.488262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.489867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.507120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 295/581] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.508223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.535468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.603451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724ff3a-02d6-4488-9db7-2036dda282cc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.619729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09422952-2757-452f-b93b-41ef6161c343 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.640262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.641121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.641410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.641689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.648349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.649832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.656631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.657330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.657645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.657881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.658656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.658965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.660528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:37.663809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:37.664716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:37.665623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 287/582] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:37.666112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.678898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 296/583] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.678898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.691124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.771928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.776768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885b603-a7cf-43e8-bde0-0df765c5a76e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.794870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62847f3-29bb-4b21-9532-79f6f82fe3dd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.812398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.813790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.814130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.814482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.815266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.815578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.817039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.831235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.832028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.832316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.832629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.834416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 288/584] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.835207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.846275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.847758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:37.863004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:37.863946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 297/585] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:37.864916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:37.916957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:37.943234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290f071-27a5-413b-9e71-b56cbcd4973d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.973445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08037d7-68bf-4d66-8851-e13a66f91d8d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:37.977816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:37.978479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:37.978763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:37.979040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:37.979824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:37.980135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:37.981601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:37.998158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 289/586] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:37.998158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.014491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.015305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.015576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.015855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.027643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.029146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.031810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:38.050552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:38.051519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 298/587] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:38.057081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.116574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63ffaa-13cc-4b61-b06d-e0a40a62ae10 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.152335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.153192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.153481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.153766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.155674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.165804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.167282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.173231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70df3-3a7e-4b17-b51a-3d6ad66cd4aa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:38.185539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:38.186434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 290/588] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:38.188643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.215193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.215732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.216014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.216289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.217143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:38.217455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.218948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.238094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 299/589] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:38.238919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.298466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.306250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15e9d8-ab4d-4636-be24-da7b75653791 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.341499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.342375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.342667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.342935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/36672cdeb2f148698b116e06095955d2/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.349248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39292195-557f-42b9-9ff3-19a3b9127597 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.358142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=36672cdeb2f148698b116e06095955d2, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.359591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.376708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 291/590] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:38 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/36672cdeb2f148698b116e06095955d2/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:38.376708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.406091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.407873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.408256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.409157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.410252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:38.410694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.412276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.430649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 300/591] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:38.432555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.432996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.516975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3645b0-de1c-49c0-86c8-39562f4c8f3b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.568505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.569642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.570133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.570639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.571620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:38.572289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.574073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.580125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696c57f-52e3-43de-84cd-c3d7190eb774 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.589492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 292/592] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:38.590717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.596408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.620624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.621436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.621720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.622005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.635004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.636586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:38.655756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:38.657572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:38.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:38.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:38.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:38.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:38.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 301/593] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:38.659139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.704071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.712406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6224bc6-7225-4e60-8764-c025ec9adfa0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.754867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.755676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.755969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.756252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.764574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.766042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:38.781440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:38.784225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 293/594] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:38.786435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.791670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7f68-69ed-457d-aa5c-d2c264049a88 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.834208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.835228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.835614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.836012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.843650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.845782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:38.861293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:38.862987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 302/595] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:38.865032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:38.893670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:38.917713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06febe-326f-4a4e-babd-e40e5bcb3474 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.968428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:38.969914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:38.970379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:38.971057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:38.971941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:38.972661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:38.974955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:38.982885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6118b-1dcd-4a38-a472-c93b9e396493 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:38.993470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 294/596] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:38.994640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.023658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.031140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.031920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.032413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.032820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.033772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:39.034191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.035794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.054839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 303/597] 10.222.0.41 () {56 vars in 1139 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:39.055723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.145704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb2b31-fd71-4845-a5ba-6fabba248849 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.150560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.169343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559f370-7339-49fe-a79e-78e81ac8aa40 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.191013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.191681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.191964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.192240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.193064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:39.193376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.194807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.208103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 295/598] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:39.208931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.217269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.218364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.218364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.218636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.226430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.227877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e30b4cf0-ccb0-4eb2-b224-df5c968784c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:39.242141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:39.243050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:39.243940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 304/599] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:39.245311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f521a2-8958-4fe0-8702-8ec2de9fe1dc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.245311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f521a2-8958-4fe0-8702-8ec2de9fe1dc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.245311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f521a2-8958-4fe0-8702-8ec2de9fe1dc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.250523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 305/600] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:39.251299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f521a2-8958-4fe0-8702-8ec2de9fe1dc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.298674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.327592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fbdb3-9fa7-4c4b-8218-e2730255d997 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.360429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f521a2-8958-4fe0-8702-8ec2de9fe1dc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.361937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.362734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.363020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.363296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.377388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.378858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:39.398177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:39.399058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:39.399969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:39.399969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:39.399969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:39.399969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:39.399969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 296/601] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:39.400946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.402951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.404175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.404476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.404795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/c1a532f2e8b446b7a03b38b3dda377a9/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.420336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=c1a532f2e8b446b7a03b38b3dda377a9, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.421822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82599ef-357e-478b-9844-872d3e167adf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.424820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.444475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 306/602] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:39 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/c1a532f2e8b446b7a03b38b3dda377a9/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:39.444475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.444475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.444475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.468260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:39.470214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:39.527466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614c597-2c29-46c9-90bc-5dcf482aa7c5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.549548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 307/603] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:39.550598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.559035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.568508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.569436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.569837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.570199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.571221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:39.571631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.573209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.588551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 297/604] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:39.593776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.699802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf93f9c-2c90-4a21-9061-ff5bb2f12526 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.722837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bfcc2-f9e5-4d20-86fd-095f8f0857b4 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.733602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.761148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.762048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.762461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.762879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.763828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:39.764144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.765563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.766573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.767775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.768744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.769210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.777934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 308/605] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:39.783461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.788038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:39.798791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:39.799925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:39.801003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:39.803614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 298/606] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:39.809358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:39.880935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:39.895948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765307d5-9737-4f75-9c29-19e3e53f6d52 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.933980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:39.934901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:39.935198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:39.935687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:39.951583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=f2d44da5ce124bd5b18462136f91fcb6, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:39.953115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:39.960679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483f3e0-f00e-41b5-8335-04a748235234 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:39.975473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 309/607] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:39 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:39.976614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.023279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:40.038554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.039389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.039673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.039959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.057946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.059487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:40.094790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:40.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:40.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 299/608] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:40.105083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.115888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3493972-2027-496b-9a67-568f57327d23 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.156277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.158482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.159224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.159706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.161204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:40.161801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.163498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.174927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 310/609] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:40.181052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.307251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ee4f-1482-44d6-a1df-fde64195b01a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.316865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:40.347853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.349003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.349516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.350010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.350349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fedca2-e0fc-4912-9102-c4e5cbbc541d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.363066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.364885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.374679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:40.394107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:40.394991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:40.395855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:40.395855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:40.395855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:40.395855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:40.398038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 311/610] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:40.403111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.441327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.443106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.444632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.445681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.446625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:40.448148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.450978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.473714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 300/611] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:40.481432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.549861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f0d8-66a9-4886-94d7-d21251687152 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.588710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.589575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.589866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.590150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.590900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:40.591217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.592689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.606641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 312/612] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:40.612617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.636822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:40.697894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5427c76-7b6a-4027-93e8-4769a4d35473 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.745114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554f661-7257-44a8-9529-0946a4186b4b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:40.759252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.760225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.760541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.760821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/9f3ecb0e259e4ed89f354e0e95927f10/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.784096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:40.784938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.785221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.785492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.789846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=9f3ecb0e259e4ed89f354e0e95927f10, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.791548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b2242-308d-41d6-8e4f-4775eed94ba8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.797770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:40.799266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:40.816768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 301/613] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:40 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/9f3ecb0e259e4ed89f354e0e95927f10/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:40.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:40.818049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:40.818971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 313/614] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:40.821721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611754-117a-4f43-886f-ecda55b1a80b None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.822458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611754-117a-4f43-886f-ecda55b1a80b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.823249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611754-117a-4f43-886f-ecda55b1a80b None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.824185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:40.825514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 302/615] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:40.837667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:40.839032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:40.839524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:40.873458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:40.874676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:40.912835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:40.948772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.003507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595b050-0f01-4eef-85e0-96ff4621fe0d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.042348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 303/616] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:41.068861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.071335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.072136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.072550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.073638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.073914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:41.074327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.075923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.096303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 314/617] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:58:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:41.103399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.242631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ce84e-9d2e-4762-96ec-3671407b0da9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.277050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0f5e6-763c-4538-8e84-27176b7a9e56 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.305537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.313206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.314224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.314515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.314800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.315116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.316109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.316399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.316862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.327523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.329668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.349477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:41.351883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:41.352770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:41.353632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:41.353632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:41.353632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:41.353632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:41.353632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 315/618] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:41.359374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:41.360883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:41.362470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 304/619] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:41.364056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.369861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.385354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.487866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eca9c-18d2-4218-b853-6f5ad229a474 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.526051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.527124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.527609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.528099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.529084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:41.529687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.531283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.542107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 305/620] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:41.549400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.625890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.655552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e96f6-47f9-4d60-9ebf-e6dc31ce8638 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.662515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ebf4-4e0b-438a-9201-6fbb02f3a586 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.699745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.700914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.701521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.701994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.714557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.716311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:41.735014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:41.735934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:41.736854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:41.736854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:41.736854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:41.736854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:41.739289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 306/621] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:41.744825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.744945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.745530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.745865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.746082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.746814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:41.747124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.748534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.778974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 316/622] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:41.786400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:41.867431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f98a-8638-420b-b45c-47ad06e75741 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:41.885614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.894314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:41.913782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:41.914803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:41.915314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:41.915798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:41.916750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:41.917258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:41.918863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:41.949905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 307/623] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:41.952238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.011907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d335-e562-4963-a3bb-4316d0e6f731 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.083441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.084256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.084570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.084852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.092716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.094901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.107170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303312e7-4423-4509-9812-26e0d6c61b05 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ceeca8fc-37f3-4a0b-8b30-998907d5dd44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:42.110960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:42.111834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:42.112830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:42.116760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 317/624] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:42.118260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d9e8b-f643-4b34-b6f4-992adf2f906c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.118534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d9e8b-f643-4b34-b6f4-992adf2f906c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.118817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d9e8b-f643-4b34-b6f4-992adf2f906c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.120635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 318/625] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:42.121415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d9e8b-f643-4b34-b6f4-992adf2f906c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.149485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.150649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.151150 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.151620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.160415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.162171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-344792fb-523c-440e-94ec-fddee18bb9c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:42.177379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:42.178372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 14:58:42.179324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:42.181930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 308/626] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:42.184533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.188607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.189050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.206685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:42.208050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:42.243313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d9e8b-f643-4b34-b6f4-992adf2f906c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.247692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.248819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.288556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 309/627] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:42.288556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.296515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.297205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.297544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.297813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.298618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:42.298929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.300402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.319704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 319/628] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:42.324576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.410059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0009dba-1bcd-446a-a927-df8016066708 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.446216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.446850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.447135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.447412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.448246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:42.448590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.450096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.453359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619d51a-7f0a-42af-8f4c-d96c3c66680f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.475066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 310/629] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:42.476091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.496610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.497808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.498352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.498830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.510767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.512921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:42.523011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:42.523951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:42.524818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:42.531747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 320/630] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:42.532945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.555354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.585967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c85a13-6955-454a-b954-df9c500fa3a8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.622064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.623135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.623420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.623697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/5b7e595d067a42dca56f0ef5fe8485c1/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.639941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=5b7e595d067a42dca56f0ef5fe8485c1, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.642572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.659713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 311/631] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:42 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/5b7e595d067a42dca56f0ef5fe8485c1/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:42.661593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.668744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba57f7f-fd16-4108-a652-ae1898cbc337 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.721167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.722640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.723098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.723977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.771641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.796065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb41e-8ef5-4fe9-8c62-af77edc4607a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.829828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.832726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78581f-6bc3-4c00-b2dc-c2120d74e9e3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.836436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:42.837241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.837590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.837934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.838756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:42.839127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:42.840931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaea8b-d653-4616-94e5-8708818ee4fd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:42.855518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 312/632] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:42.855518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823401-1b35-471a-bac3-397514302001 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.855518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823401-1b35-471a-bac3-397514302001 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.855518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823401-1b35-471a-bac3-397514302001 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.859376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 313/633] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:42.861436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823401-1b35-471a-bac3-397514302001 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:42.863772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:42.967339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 321/634] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:42.972257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:42.972257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:42.972257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:42.994626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:42.995403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823401-1b35-471a-bac3-397514302001 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:42.995854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:43.031626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.033001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.033628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.034204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.045804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.048228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.050591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:43.068729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.069673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.070554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:43.070554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:43.070554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.070554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:43.072580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 314/635] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.077825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.079413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 322/636] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:43.084558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.163517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.207406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a773349-7a61-4803-83e0-487797d7a2fd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.240090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c2ecf-bc26-4546-b461-61916a3d13e8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.246320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.247110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.247390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.247664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.259909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.261272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:43.279240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.280189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.281111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:43.281111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:43.281111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.281111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:43.281111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 315/637] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.281393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.297070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.299427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.299889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.300886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.302066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:43.302523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.304859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.326317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 323/638] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:43.326317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.347999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.416687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4dc0c-8453-480a-a814-c4515bedf214 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.436207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc812f-2e18-4888-a915-bb17949f3343 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.445492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.456212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.457046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.457333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.457612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.470849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:43.475075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.475750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.476039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.476321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.477077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:43.477396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:43.478513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.479400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:43.486168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:43.486805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604c3db-96dc-43a5-b965-e0b80fb302b1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.486888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 316/639] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.486888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.496156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 324/640] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:43.497505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d76d-fd49-4f5c-a83f-0cba7448f278 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.497800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d76d-fd49-4f5c-a83f-0cba7448f278 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.498073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3d76d-fd49-4f5c-a83f-0cba7448f278 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.503344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 325/641] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:43.503344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3d76d-fd49-4f5c-a83f-0cba7448f278 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.599818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f337ce4-cbe7-43ab-899b-a3ce20f937a4 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.607469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.613275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3d76d-fd49-4f5c-a83f-0cba7448f278 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.638045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.638840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.639126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.639521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.648248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.648805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.649471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.649724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.649825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.650019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.650760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:43.651071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.652598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.661228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 326/642] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:43.662020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-f9ef84a4-f975-4be1-a83e-55ad89701b6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:43.665672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.666758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 317/643] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.668095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.668771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.668771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.687363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:43.688805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:43.699089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.759935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 318/644] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:43.760732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.783991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606afa3-df1c-4ca7-9f09-b94cc9e56d15 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.818559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.819355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.819639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.819914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.826886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.828339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:43.844242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.845965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 327/645] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.877233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.885343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80779747-7b9d-4359-9408-1b375bda1b76 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.928944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.930238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.931206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.931705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:43.945970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:43.946260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:43.947794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:43.958117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb04e-be42-42ce-b8ed-a12ccecefbfb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:43.967978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:43.968926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:43.971131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:43.971131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:43.971131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:43.971131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:43.971131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 319/646] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:43.972909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:43.997622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:43.998581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:43.998872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:43.999152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.000709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:44.001028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.003021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.031245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 328/647] 10.222.0.41 () {56 vars in 1137 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:44.031245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.078775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a774-4464-443d-a4d2-98ecf92a1417 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.121736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.122804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.123093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.123516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.134656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:44.139274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2c9a-5498-425b-93cf-ce90a8814252 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:44.142363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:44.143504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 320/648] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:44.144352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.156215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.183606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.184269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.184584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.184869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.185650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:44.185965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.187412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.199826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 329/649] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:44.200649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.213741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.263226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e4a08-fd67-4ac3-85fd-024f6586a7f8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.299601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.300188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.300496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.300780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.301500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:44.302240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.303627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.315998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1c803-7c2a-4d65-8451-a99e1d439b03 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.321197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 321/650] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:44.321197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.356258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.357054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.357344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.357624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.395834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.435043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.436545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.438763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243af8f4-d1cf-4220-ac03-99879b421ed7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.447084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.479634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.480304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.480635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.480914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.481660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:44.481971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.483342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.492477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 322/651] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:44.493297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.552108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 330/652] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:44.552108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.611027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7a1b9-11ea-442d-8ea2-b0d592bb064a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.644108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.651085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.651865 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.652159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.652552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.653479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=36672cdeb2f148698b116e06095955d2) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:44.653802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.655353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.660185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3aae-4216-460a-8efb-bb8d64e5cad8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.662837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 323/653] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/role_assignments?user.id=36672cdeb2f148698b116e06095955d2&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:44.663651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.678428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.717299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.718729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.719533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.719950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.730651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.733470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:44.756948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:44.757986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:44.759033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 331/654] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:44.759769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.803947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01765-3aa6-4d24-aab9-f36a89fa423a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.858448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.859380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.859735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.860259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.875291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.877994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.883021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce61d95-aae0-4461-9c10-6d5d5112518b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:44.899832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:44.900797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:44.901672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:44.901672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:44.901672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:44.901672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:44.902527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 324/655] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:44.902674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.930373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:44.931180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:44.931467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:44.931746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:44.944998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:44.946353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:44.966580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:44.967724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:44.968659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:44.968659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:44.968659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:44.968659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:44.968659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 332/656] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:44.968933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:44.973274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:44.995641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.034881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32724216-21ae-4920-9618-6d2f93b32c94 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.073020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.073893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.074183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.074463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.075954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:45.076417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.078010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9fbdd-c52b-4feb-9963-32e7b7ede723 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.078339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077737ee-8db3-4cbf-83e7-1fa9fabfaa54 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 325/657] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc997e-bb44-4744-b32b-38145e7fd0d9 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc997e-bb44-4744-b32b-38145e7fd0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc997e-bb44-4744-b32b-38145e7fd0d9 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 326/658] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:45.103303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc997e-bb44-4744-b32b-38145e7fd0d9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.124240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.125075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.125198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.125492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.126767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:45.127078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.128724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.140126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 333/659] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:45.140978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.219068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc997e-bb44-4744-b32b-38145e7fd0d9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.221999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.226331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.247137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ef96b-02a8-4c2b-aeda-5b0908c3ebfa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.261016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.261804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.262087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.262364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.263097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:45.263394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.265048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240abfd-e40e-41ef-9f5b-19a61ba94c39 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.278019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 327/660] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:45.279366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.279662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.279942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.289152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.289929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.290317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.290698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.296972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:45.299022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:45.363444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.364862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.373542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 328/661] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:45.374616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.474632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.476044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.484044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f948f-eec5-42e8-9ea9-fe80a22949f4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.501947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 334/662] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:45.502885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.522748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.523829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.524181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.524558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.532125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.534138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:45.550115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:45.551198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 14:58:45.552299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:45.558494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 329/663] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:45.558494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.616663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76097b65-39ca-47df-b3c3-caae4ad70cba None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.653369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.654372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.654817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.655121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.662325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.663483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7f9d-1153-465c-ada4-957aa6b5973c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.663794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:45.677935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:45.678834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:45.679710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 335/664] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:45.680615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.706160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.706955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.707243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.707519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.719145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:45.727735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:45.728688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:45.729522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 330/665] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:45.730201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.738074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.748224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.810917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883448b9-f189-4272-99e5-a6fa97cddfdb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.841756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0183-96a0-4b46-b5cd-900e1cc3f744 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:45.854329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.855122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.855409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.855688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.867927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.869355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.880139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:45.880878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:45.881168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:45.881445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:45.882333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:45.882619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:45.884099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:45.890016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:45.890979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 336/666] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:45.897177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.906859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 331/667] 10.222.0.41 () {56 vars in 1139 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:45.908411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:45.990255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:45.999780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.010605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3361f2-a7c9-454a-84fb-e3c1d2027021 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.025563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4316ad3-9ef5-4aaf-be1a-58e279865204 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.048579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.049410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.049700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.049982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.050769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:46.051149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.052688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.062868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.064469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.064783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.065060 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.070487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 337/668] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:46.070640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:46.070640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.070817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.070903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.078478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 332/669] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:46.079537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.185037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f90aa-e3b1-4d5f-b0a9-d6ff0682b752 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.192993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329ab40-2c2e-4bc5-a842-11afe0969395 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.222898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.223569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.223874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.224138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.224890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:46.225204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.226596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.229382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.230029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.230315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.230595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.233798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.244092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 338/670] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:46.244092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.261567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.320428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.322503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.362314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ab783-d191-460a-bbcc-0394b1a10239 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.405080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.405741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.406031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.406315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.407964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=c1a532f2e8b446b7a03b38b3dda377a9) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:46.407964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.408960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.415203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 339/671] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/role_assignments?user.id=c1a532f2e8b446b7a03b38b3dda377a9&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:46.415999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.462846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 333/672] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:46.464086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.506682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.526215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.554111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2dd9e-bfa8-4ed6-a62d-6509f68434ba None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.597573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.599334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.599827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.600547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.604876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9475e6-5064-4444-ae85-fb4054110fff None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.615828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.617815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:46.642115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:46.643189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 340/673] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:46.645776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.658333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.659647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.659940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.660221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.668699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.670171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:46.685719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:46.686608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 334/674] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:46.687763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.768113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c7a6b-3555-4b8c-8513-d406b3629556 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.796629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.806668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.807606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.807967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.808319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.820627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:46.822092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.823901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.829718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c2c16-c762-44c0-9ace-a444a86cb910 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:46.844720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:46.845726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 341/675] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:46.847025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.880176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:46.880873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:46.881160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:46.881442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:46.882172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:46.882483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:46.883866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:46.897534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 335/676] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:46.898256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:46.983827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88496a06-a0f9-4671-9064-855a414f5226 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.020735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.021329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.021617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.021960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.022992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.023310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.024797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.028746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ac35-4f1f-4f3a-825f-fac6b4bca7ce None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.039972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 342/677] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.040896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.067468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.068495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.069000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.069476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.070394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.070880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.072437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758dc69-0723-4774-995f-7c6c906f60f6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.084904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 336/678] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.086290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.087250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1933eb-aa32-4efd-9867-6a77a79435b9 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.087543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1933eb-aa32-4efd-9867-6a77a79435b9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.087821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1933eb-aa32-4efd-9867-6a77a79435b9 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.093820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 337/679] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:47.093820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1933eb-aa32-4efd-9867-6a77a79435b9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.109803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.159159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcde85-9cbb-4fdc-8e3a-cf452ae75f47 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.196543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.197347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.197633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.197918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.201639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1933eb-aa32-4efd-9867-6a77a79435b9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.205501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.206961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:47.225030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:47.226035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:47.226978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 343/680] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:47.228196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.244874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.245927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.246490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.246783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.247773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.248095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.249853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d2f9b-376d-4aac-a949-5bc79a7192c8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.259419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 338/681] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.260868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.261234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.261634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.281012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:47.282202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:47.365000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.371062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f66ef-dc75-4ef4-bae5-d1d3f043b5f1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.388598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 339/682] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:47.390098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.403767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.443306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.444522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.445005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.445484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.453986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.456521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:47.471516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:47.472415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:47.473309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:47.475762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 344/683] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:47.476865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.524316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae182e9-d0d2-46af-ae89-8d4d854ce941 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.578136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.579640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.580075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.580626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.582438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.582438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.583482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.604144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 340/684] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.605109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.626494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9120-6430-4010-85a0-583ae5101f69 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.676298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.677306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.677598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.677882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.692484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.693149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.694290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:47.718131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:47.719246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 345/685] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:47.720764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.745488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.748436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacbc4b-f475-4115-8c46-f399e4aea100 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.789082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.789851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.790338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.790693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.791662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.792050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.793672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.807561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 341/686] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.808625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:47.836873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed697df-515e-4900-82f4-295964b6278a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.873736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.874512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.874862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.875208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.925452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ba41f-6fba-4ae5-83f3-f5511bf9bac6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:47.947916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.949417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.953959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.964221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:47.965061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:47.965986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:47.966336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:47.966672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:47.967733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:47.968344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:47.970718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:47.989567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 342/687] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:47.990472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.056895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 346/688] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:48.057735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.104762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e9de7-3392-4f84-8dc9-a8fdc06409fd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:48.145685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:48.146402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.146692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.146969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.147941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=9f3ecb0e259e4ed89f354e0e95927f10) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:48.148331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:48.149814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:48.157085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 343/689] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:47 2022] GET /identity/v3/role_assignments?user.id=9f3ecb0e259e4ed89f354e0e95927f10&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:48.158368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.166936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02086216-3a8b-4b1b-bba5-c5fc03f18de5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:48.183254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:48.208655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:48.209456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.209836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.210186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.210983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:48.212686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:48.223694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:48.248960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 347/690] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:58:48 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:48.311793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3464b54-0eb6-4371-acee-f8dc132eb63b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:48.366007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:48.366763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.367045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.367324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.391024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:48.393495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:48.420003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:48.422135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:48.424136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:48.424136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:48.424136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:48.424136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:48.424136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 344/691] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:48.427399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.545210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86b1c9-e0da-4c83-8336-3ba90c48995c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:48.583338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:48.584420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.584929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.585418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.586344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:48.586853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:48.588427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:48.603951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 348/692] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:48.609859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.729692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb9e1-208c-4705-9dda-5b2cbbbcfc97 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:48.731250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:48.767199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:48.768003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.768286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.768595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.775706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:48.777190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c2f6bb7c-515f-4dfa-a4b1-ee0a6094a6c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:48.792877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:48.793805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:48.794686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 345/693] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:48.800410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:48.851649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:48.878692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca8348-d6ec-489a-ae55-dec9ed1dc0f0 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.878986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca8348-d6ec-489a-ae55-dec9ed1dc0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.879434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca8348-d6ec-489a-ae55-dec9ed1dc0f0 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.881199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 346/694] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:48.889081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:48.889449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:48.889828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:48.914385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:48.915680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:48.950676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecefc76a-d0c3-49e5-93f0-3ae66ffcc428 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:49.015998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:49.016688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.016951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.017227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.018015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:49.018327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:49.019895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:49.034063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 347/695] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:49.044975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 349/696] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:49.051424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:49.057336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:49.160877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:49.168493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ed39a-cf48-4059-9f81-2554f98c5932 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:49.209147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:49.210918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.211379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.211828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.212900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=5b7e595d067a42dca56f0ef5fe8485c1) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:49.213448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:49.215727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:49.223184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 348/697] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:49 2022] GET /identity/v3/role_assignments?user.id=5b7e595d067a42dca56f0ef5fe8485c1&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:49.262125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:49.279190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20f184-29fe-4fdc-91e2-35dc7320357e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:49.318844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:49.319665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.319948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.320229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.320922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:49.322314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:49.337401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:49.338425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:49.339258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:49.344005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 350/698] 10.222.0.41 () {56 vars in 1129 bytes} [Fri May 27 14:58:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:49.346284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:49.487176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0454403-2339-4843-a2f2-7e854b47b893 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:49.534573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:49.535184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.535469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.535751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.536485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:49.536803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:49.538187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb152-4077-46df-9dac-347f829380a5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:49.543807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 349/699] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:58:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:49.552916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:49.673965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc4c46-ee8e-46aa-a391-d9d6f98a0bec None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:49.712166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:49.712868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.713155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.713435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.714149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:49.714461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:49.716109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927e064-cef4-4485-94b8-c489ffcb19d9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:49.722294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 351/700] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:58:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:49.871171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd86d0-f455-4454-bfe3-e06760464567 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.871517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd86d0-f455-4454-bfe3-e06760464567 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.871739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd86d0-f455-4454-bfe3-e06760464567 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.873565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 350/701] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:49.880526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:49.880855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:49.881104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:49.901823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:49.903049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:49.924832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:50.007669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 352/702] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:50.035767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd86d0-f455-4454-bfe3-e06760464567 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:50.070588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:50.186356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd86d0-f455-4454-bfe3-e06760464567 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:50.226002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:50.226805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:50.227086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:50.227357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:50.243448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:50.251290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:50.252220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:50.253496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 351/703] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:50.260397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:50.398651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf0f5a-55a4-4d09-a6d7-ac4f3c1043d2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:50.569957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8031-5040-43af-b218-746c673d9451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:50.570637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8031-5040-43af-b218-746c673d9451 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:50.570920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8031-5040-43af-b218-746c673d9451 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:50.571198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8031-5040-43af-b218-746c673d9451 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:50.571948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8031-5040-43af-b218-746c673d9451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:50.572256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8031-5040-43af-b218-746c673d9451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:50.573684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8031-5040-43af-b218-746c673d9451 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:50.583025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 353/704] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:50.588652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:50.704727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:50.737706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc9f7-95ea-4993-9e19-e317b2df86a6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:50.789561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:50.790363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:50.790648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:50.790930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:50.815360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:50.816931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:50.840155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:50.841062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:50.842336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:50.842336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:50.842336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:50.842336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:50.842336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 352/705] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:50.848680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8031-5040-43af-b218-746c673d9451 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:50.871083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:50.991036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8031-5040-43af-b218-746c673d9451 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.026763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.027494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.027780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.028062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.028830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:51.029139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.030570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.049041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 354/706] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:51.054748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.191306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca965-75ec-45df-bc77-ff54795e57ef None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.231547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:51.242153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.242972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.243255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.243528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.255528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.257090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c84f1cf1-3f7f-48cb-88f9-910afd98d095 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:51.276245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:51.277266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:51.278617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 353/707] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:51.285297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.290615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf64d0-7a5b-49e4-aa9e-a08db9d42445 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.290908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf64d0-7a5b-49e4-aa9e-a08db9d42445 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.291184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf64d0-7a5b-49e4-aa9e-a08db9d42445 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.292824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 354/708] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:58:51.299309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.299599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.299882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.325265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:51.326480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:51.429013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:51.463803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 355/709] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:58:51.483182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510d0ac-494e-44ca-8493-4abb2de4e7e8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.501031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.540243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.540938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.541227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.541504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.542342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:51.542655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.544108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.579862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 355/710] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:51.586778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.647057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95649750-16d2-4045-b798-1f8908dc7144 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.674083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:51.687938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.689230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.689801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.690317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] PATH_INFO: `/v3/services/network` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.691381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.693127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9320397a-4ad8-47d2-b673-814d6693f77e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:51.712830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:51.713903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:51.715008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:51.716396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0b6c8-7945-4ea4-a3e4-93e22cbaa6c3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.724085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 356/711] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:58:51.726024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a93885-9a95-401d-bd45-3d251cd13ec6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.726549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a93885-9a95-401d-bd45-3d251cd13ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.727046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a93885-9a95-401d-bd45-3d251cd13ec6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.729017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 357/712] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:51.729955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a93885-9a95-401d-bd45-3d251cd13ec6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.746760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:51.758973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.759635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.759919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.760200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.854742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a93885-9a95-401d-bd45-3d251cd13ec6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:51.857186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.859574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.896244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:51.897390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.897873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.898434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.899444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:51.899970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:51.901621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6131-dd9e-4ead-8a4b-1054e89d47e9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:51.911428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 358/713] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:51.913239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:51.913734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:51.914235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:51.933162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:51.934801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:51.985608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 356/714] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:51.986683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:51.988575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.015374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 359/715] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:52.016910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.028465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.120276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a6d33-e725-4d18-8924-d49eb7d3fe12 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.153515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9acff-03aa-4f6d-8471-93f32cd75ef9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.165146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.166166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.166282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.166522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.167276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:52.167589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.169222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.181011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 357/716] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:58:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:52.182079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.212408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.213685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.214180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.214659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.238913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.240770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:52.264620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:52.265637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:52.267141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:52.267141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:52.267141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:52.267141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:52.269596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 360/717] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:52.281536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.326651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.337593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c613b58-df15-4eab-883c-c2545cd83d2f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.343143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.383164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d8e43-d439-436e-b921-985127718760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.383974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d8e43-d439-436e-b921-985127718760 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.384257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d8e43-d439-436e-b921-985127718760 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.384890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d8e43-d439-436e-b921-985127718760 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.396317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d8e43-d439-436e-b921-985127718760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-866d8e43-d439-436e-b921-985127718760 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:52.404225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:52.405292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:52.406142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 358/718] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:52.407840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc0de-955a-4f41-895b-44225aa4812c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.411028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc0de-955a-4f41-895b-44225aa4812c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.411028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcc0de-955a-4f41-895b-44225aa4812c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.414129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 359/719] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:52.415068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcc0de-955a-4f41-895b-44225aa4812c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.433925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364226f-2a28-4e59-9c70-2b9ea8d8e4e6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.472191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.472949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.473235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.473515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.474297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:52.474610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.476048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cd0b-397d-4539-8be2-c9a65fb19468 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.488690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 361/720] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:52.489990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801013d-e2c4-44c9-ba9a-5a017b73b2c3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.490232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801013d-e2c4-44c9-ba9a-5a017b73b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.490513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801013d-e2c4-44c9-ba9a-5a017b73b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.507576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d801013d-e2c4-44c9-ba9a-5a017b73b2c3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:52.508806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d801013d-e2c4-44c9-ba9a-5a017b73b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:52.534703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcc0de-955a-4f41-895b-44225aa4812c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.571303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.571999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.572288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.572601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.573350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:52.573937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.575040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.588512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 362/721] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:52.588512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6870-941c-4a14-a74f-88faa5afa47f None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.588512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6870-941c-4a14-a74f-88faa5afa47f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.588837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 360/722] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:52.588837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.588966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6870-941c-4a14-a74f-88faa5afa47f None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.593876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 363/723] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:52.593876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6870-941c-4a14-a74f-88faa5afa47f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.609984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.621166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.701855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321924f0-8a74-47f4-9bf6-8170699df2d2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.706977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6870-941c-4a14-a74f-88faa5afa47f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.739323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.740185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.740414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.740721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.745231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.746216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.746646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.747065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.747834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.749358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.760128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:52.761623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c8bd9291-a9c6-4a4c-982e-b9ad8d5e0a7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:52.765692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:52.766814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:52.767741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:52.771662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 361/724] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:52.773012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.773306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.773589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:52.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:52.781102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 364/725] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:52.782116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.796702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:52.798491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:52.862916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.880623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 362/726] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:52.880623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.880623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:52.897622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a4d11-2bce-4204-b838-0d6da6574f4f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:52.934169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:52.934969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:52.935251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:52.935527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:52.947611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:52.955231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:52.956236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:52.957152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 365/727] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:52.957851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:52.998114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff73a8-5ac7-49d5-9866-3247c2b61142 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.036134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.037090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.037410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.037697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.038524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.038845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.040302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.058606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 363/728] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:53.058606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.072579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f429-d806-4f89-99ab-95a259a7af17 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.111860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.112870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.113155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.113541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.114263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.114571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.115954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.117463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.120332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.130678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 366/729] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:53.131651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.169568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02316ed8-d555-4e75-b584-435beca69d06 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.205721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.206384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.206669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.206949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.207674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.209089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.228628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 364/730] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:58:53 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:53.228628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.249804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014f98d-1d2d-4ee0-a3bf-b30a5e140c33 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.307215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.307913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.308198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.308494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.309263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.309643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.311182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.327655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 367/731] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:53.329301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.371219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb1b2-0bbd-4781-82f7-b58ab12425d7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.399816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.411853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.429579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.430550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.431062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.431538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.432611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.433131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.434794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.441692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 365/732] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:53.442731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.465393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b369803-9c8e-428d-a465-73c0cb68907b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.519338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.520562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.521046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.521503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.530640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.532641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:53.550500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:53.551517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:58:53.553763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:53.556154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 368/733] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:53.562199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.589325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd7231-7bb7-4aae-9fa8-4a8bd3dae036 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.639869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.641209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.641659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.655336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.658236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:53.676401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:53.677413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:53.678286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:53.678286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:53.678286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:53.678286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:53.684557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 366/734] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:53.686206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.702408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be445-0c84-4261-accc-4295d6892ccd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.724014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.733989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:53.803534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490d85a-a62e-4999-8d70-a59895b57656 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:53.805270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.805960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.806253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.806533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.807385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.807698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.809190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.845217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:53.846275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:53.846777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:53.847269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:53.848250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:53.848828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:53.850748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:53.852287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 369/735] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:53.858068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:53.867207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 367/736] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:53.873240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:54.031100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.038855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fac9c7-7898-478a-acae-331e20512e7e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.078164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.079891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.080372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.080911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.090331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:54.092086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:54.110392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:54.111368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:58:54.112297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:54.115034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 368/737] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:54.119646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf474-fc85-4f30-8847-49abc36d86ab None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.121729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:54.237924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2658a8-63a7-4c45-91bd-a2dca0448cfd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.241549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.241930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.242220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.242500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.283279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.284653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.286605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.287141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.289324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:54.291183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:54.292882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:54.332900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 369/738] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:58:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:54.338899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:54.352821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.390390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.413540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:54.415225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:54.526852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400aa36-f189-4bb9-8ace-1dfefd076c66 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.597660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.598281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.598576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.598859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.651256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 370/739] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:53 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:58:54.661475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:54.680509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:54.680509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bc83f-d098-467c-b86b-26edf2daa858 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:54.706993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.772869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.808856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 370/740] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:58:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 14:58:54.810346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c46b0-a934-4619-86ca-336a69122a83 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.810639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c46b0-a934-4619-86ca-336a69122a83 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.810918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c46b0-a934-4619-86ca-336a69122a83 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.811667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f7c67-a55a-4a77-88d4-72f6c96d2495 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.814044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 371/741] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:54.814866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c46b0-a934-4619-86ca-336a69122a83 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:54.858234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.859114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.859500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.859878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.874210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:54.877006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-88b0423c-d418-409d-ac33-9702d68f5c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:54.901290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:54.902201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:54.903402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:54.903402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:54.903402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:54.903402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:54.904560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 371/742] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:54.905833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.906123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.906407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:54.929506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:54.931449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:54.939677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c46b0-a934-4619-86ca-336a69122a83 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:54.962828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:54.987806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:54.988750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:54.989560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:54.989905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.004057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.006416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.021188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 372/743] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:58:55.022138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:55.040216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:55.043497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:55.045711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:55.045711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:55.045711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:55.045711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:55.045711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 372/744] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:55.051387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.089770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.143484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc70b-9bdf-41b5-b1fb-f7983ae328ca None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.159249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed11db-da5a-4f05-aa04-1f7dba4214d9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.180634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.181304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.181588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.181863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.182597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:55.182908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.184300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.195502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.196172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.196470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.196778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.197273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 373/745] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:55.198077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.202090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:55.202421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.204488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.217609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 373/746] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:55.218556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.249883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.312176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1957f-a06e-48d9-bf14-44b4c26b0044 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.326595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.330195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163db26f-7d95-4198-9c82-f11dd99308dc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.352553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.353256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.353545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.353847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.354731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.356326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.368017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.368864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.369150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.369428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.380551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 374/747] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:58:55 2022] POST /identity/v3/services => generated 240 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:58:55.381564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.383906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.385446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:55.414857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:55.415784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:55.416690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:55.417750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 374/748] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:55.425496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.603694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.604311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47ee00-7609-480b-bb8f-443bc09632fa None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.631374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2bf94-c9d3-424c-8ced-70ec0360ec54 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.670990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.673415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.674291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.674790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.686627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.689374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.690521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.691291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.691659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.692016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.692966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:55.693353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.694929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.709526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 375/749] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:55.709892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:55.710736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:55.711633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:55.713948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 375/750] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:55.716628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.721610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:55.761402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.837787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d8ed8-15d9-498a-a3fa-7ea150f66b23 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.859529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46fc50-4ea5-4589-a5cf-97dd36c8e408 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:55.878810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.879312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.879602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.879877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.880742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:55.881061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.882523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e715a23-3ab6-40c5-b7cf-fe79cb80ff48 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.888716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 376/751] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/role_assignments?user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:55.889960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3469c4f-f535-45b5-a4e4-9c86d18aff0d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.890247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3469c4f-f535-45b5-a4e4-9c86d18aff0d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.890525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3469c4f-f535-45b5-a4e4-9c86d18aff0d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 377/752] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28bcd-ab0d-4e11-83b2-b6eeeb1f2c1c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28bcd-ab0d-4e11-83b2-b6eeeb1f2c1c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db28bcd-ab0d-4e11-83b2-b6eeeb1f2c1c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 378/753] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.899090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.899673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.917364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:55.918591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:55.921468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:55.923715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.924266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.924731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.925701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:55.926177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:55.928560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde905a-16d4-415a-b898-e0dda4d36b6a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:55.932825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:55.946051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 376/754] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:55.947471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:55.947759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:55.948104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:55.967031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:55.968413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:56.016510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 379/755] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:58:56.016510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.098847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 377/756] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:58:56.098847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.135051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:56.157859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a589a-5028-41a7-8282-ddcb312a6435 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.195666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.196598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.197031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.197480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.198457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=5b7e595d067a42dca56f0ef5fe8485c1) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:56.198958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.200627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:56.207684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 380/757] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/role_assignments?user.id=5b7e595d067a42dca56f0ef5fe8485c1&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:56.214928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.229011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:56.240471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1633-ff54-4b5e-9ce2-386fc1ab23e5 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.277458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.278270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.278549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.278820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.284489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.284489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:56.300508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:56.301516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:56.302341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 378/758] 10.222.0.41 () {56 vars in 1129 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:56.303246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.418515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22853-4371-44bb-8ec4-e29ca397ae69 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.490275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:56.495256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.496295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.496786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.497264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.509124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:56.512583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd401d-c4c1-41be-8db3-94efa0c8c76b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:56.517922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:56.518845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:56.519661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:56.522579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 381/759] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:56.527875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.562876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.563730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.564411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.564739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.565642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:56.565864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.567323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:56.581236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 379/760] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:56.586989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.608766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:56.650411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721871f-6083-49a7-81ac-94911ff17d20 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.700257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.701257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.701724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.702212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.703136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:56.703625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.705255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:56.715759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 382/761] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:56.726425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.780670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38c79-11e6-49b5-84e0-782606bee572 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.822954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.823640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.823923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.824205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.824992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:56.825333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.826728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:56.839719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 380/762] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:56.845823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:56.862936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:56.957111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08f3b9-65f8-4b0b-a540-39d3da914c8c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:56.993789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:56.994459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:56.994745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:56.995024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:56.995819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] RBAC: Adding query filter params (interface=public, service_id=453b6663430a4895a6c92d56cf7a29e0, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:56.996134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:56.997703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.006424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650b31a-79d6-4ff5-8f90-50effdac1d4f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:57.006719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 381/763] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/endpoints?service_id=453b6663430a4895a6c92d56cf7a29e0&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:57.012956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:57.080824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:57.100847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:57.114616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:57.115484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:57.115773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:57.116073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:57.129533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf43c-5d25-46b8-bcd8-c7d87f682c3c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:57.145876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:57.147426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.173811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:57.174566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:57.174897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:57.175181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:57.175873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:57.177282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:57.182472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:57.189656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:57.190632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:57.190632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:57.190632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:57.190632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:57.190919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 382/764] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:58:57 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:58:57.193287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 383/765] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:57.201614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:57.319636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaa7c7-4da9-4e1e-9c6a-f89a1642d157 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:57.358226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:57.358831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:57.359114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:57.359393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:57.360694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:57.361056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:57.362461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.370231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:57.376552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 383/766] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:57.386278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:57.541338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf7e81-25ef-41d5-a12c-7f0bd93d3f66 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:57.578599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:57.579481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:57.579776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:57.580054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:57.588158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:57.589663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:58:57.607204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:57.608199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:57.609670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 384/767] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:58:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:57.617637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:57.739479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fd90d-b191-442f-bd9c-fcd79548c3dc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:57.784786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:57.785842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:57.786348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:57.786921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:57.787987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:57.788522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:57.790184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:57.815520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 384/768] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:58:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:57.821550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:57.910719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:58.000375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d102fff-678e-4584-90df-781cf044e442 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.055955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.056783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.057070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.057349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.081671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:58.083163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2a9b60a0-609e-490d-a59a-bfcfc2eb07cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:58.108504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:58.109459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:58.111048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:58.111048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:58.111048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:58.111048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:58.111048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 385/769] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:58.116999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.169372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2f1b4-5998-4bd7-9a5f-31c06e3cdba3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.169753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2f1b4-5998-4bd7-9a5f-31c06e3cdba3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.170137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2f1b4-5998-4bd7-9a5f-31c06e3cdba3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.171614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 386/770] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:58.191099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0b4f-e9b9-41fd-b4dc-1b74614fb14e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.191417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0b4f-e9b9-41fd-b4dc-1b74614fb14e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.191663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea0b4f-e9b9-41fd-b4dc-1b74614fb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.241797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-15ea0b4f-e9b9-41fd-b4dc-1b74614fb14e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:58.243030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-15ea0b4f-e9b9-41fd-b4dc-1b74614fb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:58.327109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:58.400559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 387/771] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:58:58.410037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82140-9204-4bf0-bf21-189a6d97dc23 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.410336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82140-9204-4bf0-bf21-189a6d97dc23 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.410659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b82140-9204-4bf0-bf21-189a6d97dc23 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.412345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 388/772] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:58.422590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.422946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.423204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.431186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a66499-3d2a-431f-a79e-3b501a65c499 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.442831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:58.444659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:58.467103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:58.472422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.473357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.473791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.474287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.475348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:58.475811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:58.477540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:58.491718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 385/773] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:58.492797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.540468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 389/774] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:58:58.540468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.629559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a59690-09aa-4430-96b2-9047e612d542 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.649160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:58.671774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce303e0-f9d3-4316-ad31-7b36a246be91 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.672563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.674416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.674869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.675295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] PATH_INFO: `/v3/services/network` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.676108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:58.677664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:58.690548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:58.691530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:58:58.692364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:58.700053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 386/775] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:58.701341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.714551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.715626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.715911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.716193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.727160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:58.729190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:58:58.749655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:58.750611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:58.752181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 390/776] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:58.757360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.777603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:58.829550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633e551-8def-4827-b035-bb67a6df4b55 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.866382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.867653 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.868128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.868638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.883312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:58.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:58.893656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:58:58.894504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:58:58.895130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73694d3-cda6-4407-9393-94d4eddd4b2c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:58.901593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 387/777] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:58.902754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.930160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:58.930752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:58.931040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:58.931322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:58.932036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] RBAC: Adding query filter params (name=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:58.932352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:58.933827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:58.942817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 391/778] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:58.942817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:58.959898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.025818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.026895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02a71a-53d7-4915-bc51-7a72e261f977 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.051773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda52c26-4371-40f0-868c-1793a287b967 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.063704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.064365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.064679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.064965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.065751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.066065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.067543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7900568-4c8b-4caa-abcb-e560ecc89af7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.084500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 388/779] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.084500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a3ce-7ba5-4860-afdb-374890f784ce None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.084500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a3ce-7ba5-4860-afdb-374890f784ce None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.084500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a3ce-7ba5-4860-afdb-374890f784ce None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.084500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 389/780] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:58:59.084908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a3ce-7ba5-4860-afdb-374890f784ce None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.096053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.096986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.097098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.097299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.098042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.098347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.099721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.108980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 392/781] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:58:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.109677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.212156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.217595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a3ce-7ba5-4860-afdb-374890f784ce None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.227245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b18a2-6c81-4561-beed-136ce9e5dfc1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.252861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.253524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.253957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.254236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.254941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.255247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.256643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1d70a-c92b-43cb-9715-81710730ccf1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.266016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 390/782] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.266016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.266016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.266016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.268034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.268951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.269235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.269509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.271505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=f2d44da5ce124bd5b18462136f91fcb6, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.273027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.282047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.285016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:58:59.286254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:58:59.296487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 393/783] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:58:59 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:58:59.296487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.375102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 391/784] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:58:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:58:59.376229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.447521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bcb38-bf48-43c2-b0f5-69e1c2caf665 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.497205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.498124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.498487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.499117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.511699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.514580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.516190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.521574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be15a59e-05a0-4c14-b62b-c9ac14c3a36a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:58:59.536570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:59.537518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:59.538390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:58:59.538390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:58:59.538390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:58:59.538390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:59.538390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 394/785] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:59.538878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.564834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.565879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.566297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.566714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.567654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] RBAC: Adding query filter params (interface=public, service_id=74c5298af73e40d0beca97d82f89eb11, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.568122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.569707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.576069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 392/786] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/endpoints?service_id=74c5298af73e40d0beca97d82f89eb11&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.577077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.612009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.669973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ea56f-0df9-47c1-b1be-b529ed5d9ed7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.709220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.710069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.710359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.710639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.720486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659083-fd86-4582-85cb-ac365e6951f0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.723068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:58:59.730908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:58:59.731846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:58:59.733120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:58:59.736572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 395/787] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:58:59.737414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.769452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.770582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.771042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.771729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.773160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.773736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.775257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.791829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 393/788] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.793384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.820700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.875734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29887927-9d20-4995-a120-2f26a1241a2d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.888822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:58:59.915125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.915946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.916334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.916920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.917646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.919135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:58:59.927533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fb50-678f-4d3d-99d3-9eefbb8ba2af None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:58:59.933062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 396/789] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:58:59.934207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:58:59.975530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:58:59.977440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:58:59.977682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:58:59.977967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:58:59.978742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:58:59.979053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:58:59.983841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.003422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 394/790] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:00.010032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.107844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e131f0f-c5ed-484f-89b5-66a75721511d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.129736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.152533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3112be8-2003-44a5-b8ae-dcfcce02c684 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.152764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.153446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.153734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.154015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.166988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.168620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-72b8d64f-060a-4cf9-9d77-e37bb8ee3b9f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:00.185131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:00.186185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:00.187141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 397/791] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:58:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:00.193540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6665-bd4b-4ebd-9882-526a316f62b0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.193540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6665-bd4b-4ebd-9882-526a316f62b0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.193540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e6665-bd4b-4ebd-9882-526a316f62b0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.193540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 398/792] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:00.195853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6665-bd4b-4ebd-9882-526a316f62b0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.205561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.206907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.207198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.207482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.222991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.225090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.240835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-539f1ee1-2a38-4bbc-bbf9-840c0effb18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:00.244805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:00.245695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:00.246562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:00.246562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:00.246562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:00.246562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:00.246562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 395/793] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:00.249143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.249143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.249143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.265587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:00.267496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:00.337245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e6665-bd4b-4ebd-9882-526a316f62b0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.337571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 396/794] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:00.338399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.403375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.404401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.404933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.405405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.406440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:00.407542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.409713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.433663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.436132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 399/795] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:00.437343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.487905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42251113-7076-400a-b1ba-b5a6179f910b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.538943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.540075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.540367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.540675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.541426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:00.541971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.543587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.551834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.561854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c845-1535-4b11-87f2-92586e9b7739 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.564519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 397/796] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:00.565803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.602704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.604539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.605054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.605711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.607075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.608750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:00.625840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:00.626820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 14:59:00.627641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:00.635155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 400/797] 10.222.0.41 () {56 vars in 1129 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:00.635155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.693776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8e5b6-fd87-4d5e-8ae7-8ab1af9f62e6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.718935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.733529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.734801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.735958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.736437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.742144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26976c6-b5fe-4f2b-b888-3e714eb673ec None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.749727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.751823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:00.770882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:00.771769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:00.772678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:00.772678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:00.772678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:00.772678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:00.774679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 398/798] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:00.775654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.777167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.777960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.778240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.778514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.787641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.789130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:00.804523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:00.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:59:00.806496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:00.810339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 401/799] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:00.811182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.831284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:00.897588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fa713-3015-4615-950e-ae85be1e2499 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.916104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3949-e885-440c-8553-3cac878547c9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:00.935890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.937148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.937636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.938131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.939824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:00.940405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.942112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.952228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:00.952389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:00.952607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:00.952839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 399/800] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:00.952936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:00.953589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:00.953899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:00.954431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:00.955679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:00.968395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 402/801] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:00.969336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.000263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.050128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.070856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f00c0b-899a-4748-9898-33b1323ad59f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.073541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97501-a5a7-4689-830f-fffe4a8a218a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.107771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.108953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.109443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.109920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.111504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:01.112066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.112238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.112896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.113179 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.113456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.114158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:01.114462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.115020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.115829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.125710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 403/802] 10.222.0.41 () {56 vars in 1124 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:01.125710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.146867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 400/803] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:59:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:01.147807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.256989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.264210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a514-2a36-4bcf-b374-cdc49a41e39c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.272668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024303eb-ed8b-4273-a547-5a4a86020152 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.309555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.310572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.311099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.311525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.312271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.312514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.313991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.314370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.314602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.315050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.322379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.324851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.330995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.340125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 401/804] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:01.341298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:01.347552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:01.348542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:01.349485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:01.352197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 404/805] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:01.360794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.494071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0a32e-aa0c-4d45-b3ad-c2b24e7e1f8f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.543587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.544403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.544711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.544987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.559401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.560931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.571622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:01.587239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:01.588202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:01.589134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:01.589134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:01.589134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:01.589134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:01.602169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 402/806] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:01.618940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.653950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28756-7c91-4e76-be55-0c9e107a848b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.705261 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.706973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.707443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.707923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.709355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:01.710113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.711760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9f530-0df4-4065-8577-557f7bd941f6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.729882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 405/807] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:01.732333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df53b-2792-46ce-b468-35cb32d82e5d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.732884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df53b-2792-46ce-b468-35cb32d82e5d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.733480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df53b-2792-46ce-b468-35cb32d82e5d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.740287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 406/808] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:01.741494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384df53b-2792-46ce-b468-35cb32d82e5d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.752508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34048dc0-a6e2-4190-a99a-38fd8e7db2b2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.789385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.790196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.790553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.790904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.791716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:01.792097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.793609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7cc5c-ab71-4135-bf56-3039dc936431 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:01.807531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 403/809] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:01.808842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.809196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.809547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.831439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.836704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:01.838434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:01.892529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384df53b-2792-46ce-b468-35cb32d82e5d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:01.915657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:01.936649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 404/810] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:01.936649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:01.964875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:01.966183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:01.966583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:01.966960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/fa4fbb1e67c24c2b8a1d816c59c75a1b/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:01.983217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=fa4fbb1e67c24c2b8a1d816c59c75a1b, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:01.984903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.003321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 407/811] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:01 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/fa4fbb1e67c24c2b8a1d816c59c75a1b/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:02.004390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.115587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10fd6-70f5-4a8f-8acf-9ce8e6b7137e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.144787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f187b-1e66-4363-b8cd-e56440d93e0d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.160877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.190415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.191432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.191814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.192195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.205236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:02.214302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:02.215703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:02.216599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:02.220639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 408/812] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:02.224747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.232536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.233346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.233631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.233911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.242966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.244476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:02.266832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:02.269689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:02.271808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 405/813] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:02.280498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.340904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.355447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b10eb6-4e00-4d0a-b05e-2b28abd03818 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.399356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.400366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.401007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.401295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.402049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:02.402411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.404643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a5851-1f49-41e5-97c2-e28e74ba065f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.420526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 409/814] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:02.423342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962a5-b04b-4bd3-b740-24d01ab171af None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.423605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962a5-b04b-4bd3-b740-24d01ab171af None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.423880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962a5-b04b-4bd3-b740-24d01ab171af None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.425516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 410/815] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:02.426247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962a5-b04b-4bd3-b740-24d01ab171af None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.434899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b61274-018b-4ab0-9bb0-1e12477a253a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.468893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.479466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.480138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.480426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.480750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.481551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:02.481865 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.483312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c9b55-fe93-4433-aba5-4ec6fa112d4a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.496894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 406/816] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:02.498245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.498541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.498821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.518253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:02.519376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:02.559127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962a5-b04b-4bd3-b740-24d01ab171af None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.600849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 407/817] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:02.601760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.608975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.611226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.611770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.612261 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.625096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.627759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:02.650240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:02.651151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:02.652026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:02.652026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:02.652026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:02.652026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:02.658745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 411/818] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:02.659642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.660409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.723966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0318dc4-6a49-4c41-be57-a16e39c34a0e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.756799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.775715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa312e-bf43-4842-9345-ce7dea5bb0fc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.776597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.778060 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.779612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.780097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/5b7e595d067a42dca56f0ef5fe8485c1/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.782684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=5b7e595d067a42dca56f0ef5fe8485c1, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.785046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.802871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 408/819] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:02 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/5b7e595d067a42dca56f0ef5fe8485c1/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:02.804081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.831496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875524-4358-41fc-9453-48fa72b035db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.832531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875524-4358-41fc-9453-48fa72b035db None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.832894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875524-4358-41fc-9453-48fa72b035db None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.833247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09875524-4358-41fc-9453-48fa72b035db None admin] PATH_INFO: `/v3/services/network` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.834045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09875524-4358-41fc-9453-48fa72b035db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.835576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09875524-4358-41fc-9453-48fa72b035db None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-09875524-4358-41fc-9453-48fa72b035db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:02.862875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:02.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:02.865008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 412/820] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:02.872033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875524-4358-41fc-9453-48fa72b035db None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:02.936259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:02.943713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0165bf8-ea63-48ca-b4be-3c5b414ca6ff None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:02.988748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:02.990206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:02.990782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:02.991341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:02.992255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:02.992741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:02.994643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.015259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 409/821] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:03.017506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.100340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09875524-4358-41fc-9453-48fa72b035db None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.142640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.143459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.144656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.145266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.145893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] RBAC: Adding query filter params (name=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:03.146309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.147805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.148367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478ac78-b02f-45d4-883d-9cee2655495c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.152885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 413/822] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:03.159496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.175602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:03.198612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.199919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.201268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.201725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.211778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.214513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:03.232530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:03.234827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:03.240725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 410/823] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:59:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:03.244383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.280846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:03.363738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704f791-9f2b-4045-af0d-c3ff971ea468 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.390897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbe1ce-7879-482d-9e97-ce873b8b435a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.404559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.406080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.406538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.406978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.408030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:03.408558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.410176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.438224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 411/824] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:59:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:03.440254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.486234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.487352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.487352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.487752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.488578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] RBAC: Adding query filter params (type=network) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:03.488963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.490484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.503422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 414/825] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:03.508826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.540854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:03.555585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bf947-0ad9-471a-9b2d-e6d828fd790a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.597024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.598139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.598625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.599102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.677186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.679219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.730748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:03.768857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:03.796881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 412/826] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:03.799621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:03.815295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b6422-aa1c-44ad-8792-96d5991eb858 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.906530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.907373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.907689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.908531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.909077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.910715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:03.911914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25438b2d-9556-4806-ba4c-8ae0af2aebad None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:03.961399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 415/827] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:03.968916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:03.970416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:03.971180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:03.971710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:03.985156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:03.987711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-0fa6c501-72a3-4def-89f5-b24240fe2688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:04.007700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:04.008726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:04.009590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:04.009590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:04.009590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:04.009590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:04.016125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 413/828] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:04.017818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.037214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb40ba-e21e-4f81-a250-c3c5667a5a84 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.037889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb40ba-e21e-4f81-a250-c3c5667a5a84 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.038481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb40ba-e21e-4f81-a250-c3c5667a5a84 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.040481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 414/829] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:04.046024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.046562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.047058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.079492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:04.082629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:04.136835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.236841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a34d85-4172-47fb-a188-154f4f27eaf6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.244874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 415/830] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 27 14:59:04.273912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.279993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.287765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.289427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.289704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.290040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.290791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:04.291100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.293184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.321602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 416/831] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:04.323686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.416080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dae55a-99b3-4425-80b6-95a1cdb103cb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.465642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.466830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.467114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.467550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.468547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.471029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.472196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f4a58-c9c9-4245-a829-6767121068a8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:04.486441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:04.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:04.488403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:04.489170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 416/832] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:04.496246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.520461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.521302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.521582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.521855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.527070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.530573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.533200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:04.549130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:04.551243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:04.553391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:04.556486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 417/833] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:04.557674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.621946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.624611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634ef12-3f25-41f9-b3fc-8d737c7cde71 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.662133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.663402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.664011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.664677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.665959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:04.666658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.669085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.683819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 417/834] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:04.685353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.686361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f13da0-f8f8-4bac-9185-3a72bd665601 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.727608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.728265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.728576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.728863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.729615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:04.729923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.731360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74c7e-dec6-4787-843c-b406e1f3d845 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.748859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 418/835] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:04.750452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e20f54-2cb2-4504-9d1b-43b989bf0dbd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.750746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e20f54-2cb2-4504-9d1b-43b989bf0dbd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.751687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e20f54-2cb2-4504-9d1b-43b989bf0dbd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.753057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 419/836] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:04.759888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e20f54-2cb2-4504-9d1b-43b989bf0dbd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:04.793599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.812205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1254140-ef70-43e3-8b1e-ebaaf68301d9 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.848069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.848802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.849085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.849361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.850116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:04.850428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.851796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612af0c3-9986-4324-b2bd-80b0830c52b1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.864474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 418/837] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:04.864474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.864474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.864474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.878209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:04.879512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:04.886919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e20f54-2cb2-4504-9d1b-43b989bf0dbd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:04.908790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:04.935664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:04.936349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:04.936664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:04.936943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:04.937796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:04.938116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:04.940241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:04.952491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 420/838] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:04 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:04.981718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 419/839] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:05.006401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.135167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495885fe-fa41-4121-9d97-8e84bd42c723 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.176955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.177765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.178051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.178328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.190167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:05.198120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:05.199075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:05.199900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 421/840] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:05.208267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.295362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:05.318970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a4e7-257c-4391-9e53-c80277e49f2e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.359066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.359799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.360085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.361079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.361869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:05.362178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:05.363570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc447e-e22c-4016-9eeb-6d4d718edce2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:05.372852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 420/841] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:05.379528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.406578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fb3-adb3-4a2a-b4c1-21236463a03b None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.406912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fb3-adb3-4a2a-b4c1-21236463a03b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.407248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fb3-adb3-4a2a-b4c1-21236463a03b None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.408924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 421/842] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:05.414084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.414375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.414662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.423407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:05.432617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:05.433813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:05.505214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb06f-c834-4ba1-b3fa-f720ec9ff15e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.524603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 422/843] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:05.547596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.553168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.554090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.554478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.554879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.566913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:05.568624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:05.588808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:05.590001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:05.591200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:05.591200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:05.591200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:05.591200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:05.591942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 422/844] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:05.597780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.627175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:05.670641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164cf7-4dc0-4e6f-b8b1-7b415d0e780c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.710869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:05.720133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.721258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.721542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.721817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.730377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218558fa-dbf0-4dda-aa07-bba5d8abc429 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.741998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:05.760623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:05.761640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:05.762471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:05.763804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 423/845] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:05.769290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.782396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.783426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.783718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.783996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.784759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:05.785070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:05.787371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:05.803022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 423/846] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:05.808278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.911177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba78d0-21f0-4841-ae0b-d4d58983eed8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.937271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6149f-46a5-4767-890c-cbf4e3d531b9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:05.944023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:05.952824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.953770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.954565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.955224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.956181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:05.956781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:05.958367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:05.967758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 424/847] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:05.973641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:05.978680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:05.979503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:05.979787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:05.980065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:05.988901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:05.990402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:06.005615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:06.006560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:06.007513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:06.011511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 424/848] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:06.012324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.023793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.107782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5d4e-f6a6-4394-bb51-35a368acf8dd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.122437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69bfe-40ca-4ca4-b405-614cd913471f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.153561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.155294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.156078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.156809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.159319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.159997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.160281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.160587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.161500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:06.161818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.163367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.176758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.179393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.190152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 425/849] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:06.191119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:06.208783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:06.209712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:06.210575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:06.210575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:06.210575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:06.210575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:06.212874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 425/850] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:06.220488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.281609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.285532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.332158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c545-7bed-46d4-9261-a210b57393f1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.357721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a01d50-f3bd-4bbe-a085-4c39f6eac248 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.377767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.379416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.380026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.380486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.402963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.403647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.403961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.404514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.405248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:06.405561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.408109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.429596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 426/851] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:06.430617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.480012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.481784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.550589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a2732-2f52-4e31-81e7-d5a6a745cf12 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.553809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.567429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.602053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.602895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.603619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.603909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.611452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 426/852] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:06.613549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.615225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.617254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:06.638748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:06.639710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 14:59:06.640661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:06.644651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 427/853] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:06.646754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.751422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31701305-bc57-48a8-a18d-49491d9066e3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.772496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58df-4aca-4357-942c-b2693b60882b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.797681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.799157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.799504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.799847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.811904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.814379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.818125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:06.818802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:06.819082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:06.819353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:06.820184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:06.820515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:06.821942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:06.836942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:06.837868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:06.838728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:06.838728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:06.838728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:06.838728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:06.846836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 428/854] 10.222.0.41 () {56 vars in 1135 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:06.846996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 427/855] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:06.847105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.851453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:06.875209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.880146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:06.963039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5673ed-5a63-4ade-ba4a-5b7bf37533cb None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:06.985850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d793e80-cbb0-4ac2-add6-eee6e98bdd23 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.004696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.005478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.006064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.006368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.007121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:07.007429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.009857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.028495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 429/856] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:07.033644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.036493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.037267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.037560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.037836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.118236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.119741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc132068-9fcc-449c-b0f1-d267b0f5df79 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.140217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.154269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.182605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25223896-a9a5-42ed-9cd5-37b220df2418 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.228505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.230086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.230842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.231324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.236823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 428/857] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:06 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:07.241362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86630-d4b6-406f-abdb-631a7a2e83df None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.241362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86630-d4b6-406f-abdb-631a7a2e83df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.241362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe86630-d4b6-406f-abdb-631a7a2e83df None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.241362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 429/858] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:07.242125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe86630-d4b6-406f-abdb-631a7a2e83df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.245041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.246518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-342fd925-c664-4cdf-9dfd-a01d87b9f413 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:07.261486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:07.262481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:07.263387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 430/859] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:07.263904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.264307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.264482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.283675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:07.284891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:07.355350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe86630-d4b6-406f-abdb-631a7a2e83df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.357702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 431/860] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:07.358681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.388240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.405436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.406229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.406512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.406787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.417427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.418878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.436687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e4269640-bdb8-4830-8afc-3d6d4d94231a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:07.436827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:07.437740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:07.438970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:07.438970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:07.438970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:07.438970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:07.440596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 430/861] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:07.441968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe4a9-a15e-4113-b16f-2a8456cfea1e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.442258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe4a9-a15e-4113-b16f-2a8456cfea1e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.442538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe4a9-a15e-4113-b16f-2a8456cfea1e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.444141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 431/862] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:07.444845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe4a9-a15e-4113-b16f-2a8456cfea1e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.469146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bc86b-e846-4fd7-b29e-29a3c650b9bb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.503814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.504400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.504716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.504994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.505781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:07.506106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.507527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.519022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 432/863] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:07.519820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.549255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fe4a9-a15e-4113-b16f-2a8456cfea1e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.584554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.585211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.585491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.585769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.586482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.587850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41c4ca-31e8-4860-9e69-b39abd90d33e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.601577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 432/864] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:07 2022] POST /identity/v3/services => generated 251 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:07.602821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.603117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.603400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.627779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b123-cb9a-461a-8c37-f9d6d4a95267 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.633776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:07.635024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:07.664828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.668412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.683406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.684590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.685135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.685682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.686603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:07.687079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.688785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.702555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 433/865] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:07.703721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.723390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 433/866] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:07.724672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.820302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e3541-5382-48fc-9b2e-0fa3eb594607 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.849755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc024121-1d2f-492c-b75f-cec2a976a922 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:07.860772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.861905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.862350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.862632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.863458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:07.863774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.865240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.871493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 434/867] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/role_assignments?user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:07.872065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.896968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:07.898510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:07.899205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:07.900140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:07.912303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:07.914280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:07.934336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:07.935296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:07.936196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:07.937022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:07.943737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 434/868] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:07.945227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:07.972957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:08.011674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ba6b-fa4c-4f3d-b162-938ac4df3dbd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:08.055469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:08.056478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:08.056860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:08.057237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:08.058068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:08.059655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:08.073815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:08.074967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:08.075843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 435/869] 10.222.0.41 () {56 vars in 1132 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:08.081334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:08.083911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d87a3-376c-4ed7-8549-5c2c591536d6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:08.132128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:08.133544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:08.134007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:08.134430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:08.135998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:08.136313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:08.137954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:08.151322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 435/870] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:08.158198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:08.212802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0739c2-fe20-4b05-996b-83756fe87797 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:08.240836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:08.252630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:08.253343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:08.254150 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:08.254495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:08.255213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:08.255574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:08.257435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:08.260960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:08.265643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 436/871] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:08.271522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:08.289579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58066e21-cea4-4488-b325-19e02b47a4b2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:08.326078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:08.326985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:08.327417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:08.327855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:08.328925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=5b7e595d067a42dca56f0ef5fe8485c1) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:08.329938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:08.331587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1816b6-9a03-403f-90c5-a35c667a9933 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:08.338485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 436/872] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:08 2022] GET /identity/v3/role_assignments?user.id=5b7e595d067a42dca56f0ef5fe8485c1&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:08.396477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4dadb-4ea9-4899-aa16-95b57489d95d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:08.432367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:08.433084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:08.433429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:08.433707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:08.434477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:08.434793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:08.436212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99480ad-6d1e-433d-8dfc-44ae9f54e70b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:08.443279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 437/873] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:09.870891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96115d-ee5f-4329-a9e2-6ba8640de848 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:09.871410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96115d-ee5f-4329-a9e2-6ba8640de848 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:09.871511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96115d-ee5f-4329-a9e2-6ba8640de848 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:09.873079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 437/874] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:09.878642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:09.879053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:09.879220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:09.897606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:09.898790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:09.942703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:09.972330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 438/875] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:09.995627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96115d-ee5f-4329-a9e2-6ba8640de848 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.111115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96115d-ee5f-4329-a9e2-6ba8640de848 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.148279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.149448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.149930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.150401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.151255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.152971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-9c863243-fb57-463b-b0ce-75f3de54035b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:10.165840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:10.166873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:10.167709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:10.170683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 438/876] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:10.177566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.276209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7703-3a34-4521-94d7-c736dac5fcc8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.276897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7703-3a34-4521-94d7-c736dac5fcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.277376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7703-3a34-4521-94d7-c736dac5fcc8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.280308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 439/877] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:10.285272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095f98-5846-4895-8b37-33bb58366553 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.285805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095f98-5846-4895-8b37-33bb58366553 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.286285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68095f98-5846-4895-8b37-33bb58366553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.287672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c52652-6070-4c5c-b3c4-05b16dc82fc2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.304432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-68095f98-5846-4895-8b37-33bb58366553 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:10.306012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68095f98-5846-4895-8b37-33bb58366553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:10.316793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.326802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.327480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.327766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.328046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.328938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:10.329418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.331126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.336304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 439/878] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:10.341777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.385932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.396580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 440/879] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:10.412921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68095f98-5846-4895-8b37-33bb58366553 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.460036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508296a1-28f5-46dd-b01b-4e816448aaad None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.501933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.502909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.503175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.503550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.504413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:10.504857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.506516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.518727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 440/880] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:10.519563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.540839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68095f98-5846-4895-8b37-33bb58366553 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.585399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.586280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.586572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.586862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.594646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.599685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:10.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:10.609583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:10.611453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:10.617817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 441/881] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:10.619237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.659395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc9b4c-0398-4f7b-be91-ba057ba2ac16 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.661421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.701549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.702231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.702606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.702955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.703996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] RBAC: Adding query filter params (interface=public, service_id=1d2ce428502044f9b1da82372af0d326, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:10.704592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.706046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008cad1-bb7b-4c45-9374-b1fac31b076b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.715777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 441/882] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/endpoints?service_id=1d2ce428502044f9b1da82372af0d326&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:10.715777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407e42-f6e4-4b31-937b-bcb7c1b1f65e None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.715777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407e42-f6e4-4b31-937b-bcb7c1b1f65e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.715777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407e42-f6e4-4b31-937b-bcb7c1b1f65e None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.715777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 442/883] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:10.716204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd15a-6921-4b4c-bc42-bc273796364a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.716757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd15a-6921-4b4c-bc42-bc273796364a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.716874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cd15a-6921-4b4c-bc42-bc273796364a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.718117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 443/884] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:10.718729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cd15a-6921-4b4c-bc42-bc273796364a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.732643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367508f4-0197-475b-94f0-7a5750cc7c68 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.766992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.767721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.768009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.768290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.769120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:10.769436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.770827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d34800-c652-45c5-ad79-1a2329068226 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.778655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 442/885] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:10.779960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307f96-880b-43c3-be2a-36c1844402ba None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.780252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307f96-880b-43c3-be2a-36c1844402ba None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.780569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307f96-880b-43c3-be2a-36c1844402ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.797281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-34307f96-880b-43c3-be2a-36c1844402ba None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:10.799120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-34307f96-880b-43c3-be2a-36c1844402ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:10.824783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cd15a-6921-4b4c-bc42-bc273796364a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:10.854575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.859033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:10.859695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.860545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.860822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.861497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:10.862864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:10.866928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 443/886] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:10.868387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 444/887] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/services => generated 1011 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:10.868538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:10.868634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:10.868884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:10.873583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:10.895177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:10.896415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:10.916580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:10.979573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 444/888] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:10 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:10.980705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.012878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906d594-13a7-4c0f-a01d-f706d4b22810 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.051596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.052399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.052710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.052990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.064093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.065581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:11.084412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:11.085396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:11.086309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:11.086309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:11.086309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:11.086309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:11.086309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 445/889] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:11.086612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.109697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f62ec-bbcc-41a0-93bd-3d7761f48d2f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.135641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.151561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.152374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.152690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.152971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.153659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.155051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:11.168073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:11.169129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:11.169963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:11.174387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 445/890] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:11.175460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.179990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.201997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef82ad-ea09-4a2c-8570-987dada63b72 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.238846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.239432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.239717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.239993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.240812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.242186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.258458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 446/891] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:11 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:11.259072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.306845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d10442-80f2-46e7-97f5-17c4aabc71f8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.353105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.353787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.354071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.354346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.355079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:11.355392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.357069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.376747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 446/892] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:11.378086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.390055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0176ab-5a28-4800-8079-b142c96a4a39 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.429023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.429611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.429730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.430015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.430297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.431037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:11.431341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.432822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.439283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 447/893] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:11.444977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.455471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.497391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557113ec-6c48-4e73-8395-ca7b8cf83934 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.540288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.541133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.541421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.541696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.552197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.553741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.565606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2f3a7-8b7c-4d80-ab08-484c4c05a224 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:11.574093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:11.575049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:11.576675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 447/894] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:11.581554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.616224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.617921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.618700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.619198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.620297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:11.620900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.622566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.629058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 448/895] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:11.706259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.709789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dff85a-e4e1-4bfd-b041-793d790dda90 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.752156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.752785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.753069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.753343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:11.754267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:11.754579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:11.756051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:11.776864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 448/896] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:11.783317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:11.860987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:11.895905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5759d-d01c-4621-ac89-d3345ebe8d80 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:11.931982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:11.933375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:11.933807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:11.934238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:12.005762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:12.007434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:12.086220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:12.114571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 449/897] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:11 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:12.120104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:12.231315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a05a74-2bdb-40db-a3a6-72f4c7c60bc6 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:12.273808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:12.275432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:12.276157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:12.276851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:12.294754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:12.296718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:12.318128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:12.319669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:12.320840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:12.320840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:12.320840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:12.320840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:12.323082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 449/898] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:12.329308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:12.433047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5546cd-cbfb-4237-9a76-912f41a3ff11 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:12.470716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:12.471439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:12.471721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:12.471996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:12.472754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:12.473061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:12.474487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:12.487067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 450/899] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:12.492472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:12.506524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:12.611875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824e6d8-6654-4d37-b7c2-0a27d6421311 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:12.649681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:12.650482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:12.650768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:12.651043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:12.658317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:12.659778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:12.674449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:12.675385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:12.676242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 450/900] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:12.682692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:12.715864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:12.788317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6d7bc-7e13-4abe-926d-69c0f97ecb14 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:12.824506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:12.825176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:12.825462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:12.825742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:12.826538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:12.826849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:12.828279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064dd27-c1df-45e6-816f-d0ce776a1802 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:12.842343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 451/901] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:12.848140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:12.923688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:12.957321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85a1db-966a-4159-83dc-9eaefc5893a3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:12.998347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:12.999029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:12.999323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:12.999605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.000582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] RBAC: Adding query filter params (role.id=44f51dd61a89463ea1de514abf8669a9, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:13.001364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:13.002922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ac07e4-b085-43bd-ae60-4da87df065c0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:13.009769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 451/902] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:12 2022] GET /identity/v3/role_assignments?user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=44f51dd61a89463ea1de514abf8669a9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:13.177117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb978c-7dcc-4902-a2fd-0334e3d7e759 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.177614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb978c-7dcc-4902-a2fd-0334e3d7e759 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.177614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb978c-7dcc-4902-a2fd-0334e3d7e759 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.179281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 452/903] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:13.185085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.185602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.185682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.203947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:13.205385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:13.284569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 452/904] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:13.304280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb978c-7dcc-4902-a2fd-0334e3d7e759 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:13.406495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:13.423775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb978c-7dcc-4902-a2fd-0334e3d7e759 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:13.459312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:13.460049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.460383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.460697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.461359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:13.462738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:13.476164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:13.477218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 14:59:13.478054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:13.482420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 453/905] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:13.483769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:13.553019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:13.598886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b7f73-1c12-483a-b565-dc77c321db9f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:13.636686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:13.637424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.637708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.637983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.638745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:13.639058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:13.640466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd00992-e783-4bb2-8700-f6ee402cbc04 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:13.650896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 453/906] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:13.652335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:13.770262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7edd6c-19da-4ca3-a21a-460eb7204b9a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:13.776308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d4b-c0e8-4a8e-bc4f-802f4e92ac02 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.776780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d4b-c0e8-4a8e-bc4f-802f4e92ac02 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.777160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b19d4b-c0e8-4a8e-bc4f-802f4e92ac02 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.778845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 454/907] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:13.784746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cdbaf-2be2-48f8-9cfb-b105ce4057cf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.785355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cdbaf-2be2-48f8-9cfb-b105ce4057cf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.785831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cdbaf-2be2-48f8-9cfb-b105ce4057cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.805670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f29cdbaf-2be2-48f8-9cfb-b105ce4057cf None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:13.807284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f29cdbaf-2be2-48f8-9cfb-b105ce4057cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:13.812164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:13.812920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.813725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.814241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.815061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:13.815371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:13.816772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:13.827762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 454/908] 10.222.0.41 () {56 vars in 1114 bytes} [Fri May 27 14:59:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:13.830109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:13.856314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:13.908794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 455/909] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:13.918742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab0d7-dae2-4226-ae41-fb63f66e4684 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.919349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab0d7-dae2-4226-ae41-fb63f66e4684 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.919978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ab0d7-dae2-4226-ae41-fb63f66e4684 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.921694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 456/910] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:13.928816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.929118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.929384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.948365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:13.949587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:13.949775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ebadf-4c38-4b95-a739-b32c0793269e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:13.991454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:13.992115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:13.992407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:13.992712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:13.993449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:13.994807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.007075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:14.014876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 455/911] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:14.015846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.044178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 457/912] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:14.067803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.118121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:14.151922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540008f-4b8e-4136-a5d3-213adc273c27 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.192911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.194585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.195091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.195563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.196434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.198093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:14.216501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:14.217757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:14.218847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:14.223286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 456/913] 10.222.0.41 () {56 vars in 1132 bytes} [Fri May 27 14:59:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:14.226517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.228867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6222-b2ed-46c7-9561-3854ff63e036 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.281443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.282215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.282553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.282839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.283823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.285719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.308480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 458/914] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:14 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:14.363423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51341a-893e-42c4-8afc-92f841c3e905 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.412875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.413551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.413830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.414101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.414811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:14.415114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.416586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.417861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:14.423984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 457/915] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:14.430010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.483690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:14.544266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303b46c-c509-4a06-b68e-78529a0a427f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.580424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.581060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.581341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.581617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.582323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:14.582681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.584063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.589591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 459/916] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:14.596599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.702626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0985f-aa1b-4b06-a10b-49dbfc9d4b66 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.740050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.740738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.741023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.741297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.742086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] RBAC: Adding query filter params (interface=public, service_id=877da47f7566449286e4cb64deea082f, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:14.742399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.743828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640de94d-1954-4d6b-87ee-e3d0f1495ead None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.749420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 458/917] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:59:14 2022] GET /identity/v3/endpoints?service_id=877da47f7566449286e4cb64deea082f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:14.755761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:14.863468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eef042-33c7-4002-91ef-abae4e1fe893 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:14.873449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:14.900718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:14.901388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:14.901673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:14.901951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:14.902640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:14.904027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f086e-66b6-46b2-befb-61858f854a13 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:14.910744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 460/918] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:14 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:15.389618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecfde-8161-454c-95f6-299b33cc2228 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:15.390100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecfde-8161-454c-95f6-299b33cc2228 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:15.390194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecfde-8161-454c-95f6-299b33cc2228 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:15.392379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 459/919] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:15.397795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:15.398201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:15.398371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:15.417157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:15.418358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:15.487614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 461/920] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:15.510249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ecfde-8161-454c-95f6-299b33cc2228 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:15.586590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:15.632437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ecfde-8161-454c-95f6-299b33cc2228 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:15.668246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:15.669017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:15.669310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:15.669593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:15.682222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:15.691115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:15.692021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:15.692897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 460/921] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:15.698221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:15.798996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b774b1-132a-41f9-8990-4d1231144ee5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:15.800597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:15.839950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:15.840906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:15.841190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:15.841466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:15.842212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:15.842514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:15.843905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:15.852906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 462/922] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:15.858323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:15.965457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da18e6-de88-4582-a9bd-af7fa23ebc2b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:15.989872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.002064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.002864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.003159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.003487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.014236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.015709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:16.035306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:16.036317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:16.037886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:16.037886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:16.037886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:16.037886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.037886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 461/923] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:16.042242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.154717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad3cf7-781e-4e60-b0ca-7d0ab5b3e5fb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.193149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.193831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.194112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.194392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.195119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:16.195428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.196857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.209603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 463/924] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:16.215068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.326209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ae27-036a-4430-87bf-d2dde15f3392 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.362309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.363127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.363414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.363694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.378469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.385072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-fd8fa73e-810d-4754-b194-a0c14e85f849 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:16.401092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:16.402437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.403928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 462/925] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:16.408634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.435464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.438423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293104e4-b6d5-4a1b-a249-b3d8bb29ddb3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.438892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293104e4-b6d5-4a1b-a249-b3d8bb29ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.439181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293104e4-b6d5-4a1b-a249-b3d8bb29ddb3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.440814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 463/926] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:16.447411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.447959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.448631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.470682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:16.472634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:16.509860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.521603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a298b7-f370-46a9-971a-113b11d6b3a8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.572475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.572475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.572475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.572475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.572894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:16.572894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.572894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.573292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 464/927] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:16.589644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.591876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 464/928] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:16.598499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.689170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.703237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea371671-2f62-49a7-9863-fb9c022c4446 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.709743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f603b-0185-4d99-aaff-d622e036f69a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.745921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.746595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.746880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.747156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.747876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.749361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.751252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.751370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING keystone.server.flask.common [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 14:59:16.752108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.752396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.752704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.763206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.764704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.768874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.sql.core [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 14:59:16.768874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 14:59:16.768874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [parameters: {'id': 'e4de46e8960c458988e38360d90802c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 14:59:16.768874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80314) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-df323599-c499-403d-9e85-66d71583fe22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:16.784212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:16.785215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:16.786148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:16.786148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:16.786148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:16.786148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4de46e8960c458988e38360d90802c3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4de46e8960c458988e38360d90802c3. May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application conn.query(q) May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:59:16.786845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application result.read() May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 14:59:16.787652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application next(self.gen) May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application yield resource May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application next(self.gen) May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self.session.rollback() May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self.force_reraise() May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise self.value May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application session.commit() May 27 14:59:16.788616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._prepare_impl() May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self.session.flush() May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._flush(objects) May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application compat.raise_( May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application flush_context.execute() May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 14:59:16.791566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rec.execute(self) May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 14:59:16.794535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application conn.query(q) May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application result.read() May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:59:16.796577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [parameters: {'id': 'e4de46e8960c458988e38360d90802c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.799490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:16.801253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e4de46e8960c458988e38360d90802c3. May 27 14:59:16.804270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:16.805070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 465/929] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:16.805168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 465/930] 10.222.0.41 () {60 vars in 1140 bytes} [Fri May 27 14:59:16 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:16.805251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15abd6e-4ca7-4482-a593-e4f79dc38f7b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.805332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.805412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15abd6e-4ca7-4482-a593-e4f79dc38f7b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.805412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15abd6e-4ca7-4482-a593-e4f79dc38f7b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.805412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 466/931] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:16.805412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15abd6e-4ca7-4482-a593-e4f79dc38f7b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.825395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.912842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f09e6e-107c-4c06-adf1-f69e8d40addc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.913479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15abd6e-4ca7-4482-a593-e4f79dc38f7b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:16.950087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.950491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:16.950756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.951041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.951284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.951393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.951583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.951835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.952030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:16.952350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.954483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c1ba0-d0d9-4c20-b858-de4f3ee0c763 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.954776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:16.961776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:16.963414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:16.976509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 466/932] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:16.976509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:16.976509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:16.976509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:16.978809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:16.979866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 467/933] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:16.980921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:16.999820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:17.002290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:17.087014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 467/934] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:17.088317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.089429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830a6ef-af15-4371-b879-9b61f82a6fed None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.108877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.133055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.135748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.136412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.136945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.147447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.149164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:17.165022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:17.166173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:17.167286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.168353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 468/935] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:17.169066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.215367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0911fb-97f3-47b2-88ef-0c3bf680b807 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.228775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.253501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.254536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.255042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.255533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.256558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:17.257080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.258713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.277434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370063-9da6-454f-a694-fc2909fc0a57 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.288814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 468/936] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:17.290576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.318742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.319568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.319854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.320131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.320851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.322288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-790ed8ab-0639-49b4-8b80-943ccbb992c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:17.335960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:17.337272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.338095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 469/937] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:17.338869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c090501-ca35-489a-b946-d2890e2059e8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.338938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c090501-ca35-489a-b946-d2890e2059e8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.339167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c090501-ca35-489a-b946-d2890e2059e8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.341197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 470/938] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:17.343643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c090501-ca35-489a-b946-d2890e2059e8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.397022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.440357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba18f-ce79-442e-9ab7-f5d47af104cc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.474584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c090501-ca35-489a-b946-d2890e2059e8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.484324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.485489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.485911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.486195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.486994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:17.487289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.488789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcae516-9487-46e1-bca5-4eb9f6b294c9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.494295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.504554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 469/939] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:17.504554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.504554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.504554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.522082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.522909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:17.523057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.523327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.523626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.524114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:17.524337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:17.524679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.526096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.532686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 471/940] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:17.533499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.598005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 470/941] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:17.599106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.660480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1ef9c-3edb-409f-98ce-f5b79ffbb3f5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.674292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.697644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.698631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.699015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.699401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/fa4fbb1e67c24c2b8a1d816c59c75a1b/roles/44f51dd61a89463ea1de514abf8669a9` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.701804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=fa4fbb1e67c24c2b8a1d816c59c75a1b, role_id=44f51dd61a89463ea1de514abf8669a9)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.703266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.720135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 472/942] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:17 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/fa4fbb1e67c24c2b8a1d816c59c75a1b/roles/44f51dd61a89463ea1de514abf8669a9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:17.721202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.735085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ced5c-6711-47bc-9de8-290045a8b52b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.784899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:17.789778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.790651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.791150 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.791542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.792943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:17.793444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.795248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.803376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 471/943] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:17.810180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.863614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf74454-f0e5-4548-a005-1abf78978230 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.905871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:17.907354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:17.907841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:17.908263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:17.909120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:17.910684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:17.927445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:17.928641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 14:59:17.929700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:17.932358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 473/944] 10.222.0.41 () {56 vars in 1132 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:17.938816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:17.953044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bc06-4ec2-4b57-8da4-a6d3ce47a6ca None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:17.994280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:18.013054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:18.013968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:18.014397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:18.014861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:18.017009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] RBAC: Adding query filter params (interface=public, service_id=53d73fafabad49c7a9b7dd64a0489170, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:18.017703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:18.019834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:18.030774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 472/945] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/endpoints?service_id=53d73fafabad49c7a9b7dd64a0489170&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:18.039422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:18.084713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823cb5-bed8-4ddb-beaf-dd0a21ec6f1b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:18.092511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:18.122631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:18.123563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:18.123865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:18.124237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:18.126361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:18.126361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:18.128035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:18.133770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 474/946] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:18.139147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:18.161532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3c0d-eb86-48b9-b49c-ed53d6785ace None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:18.200942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:18.201953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:18.202382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:18.202884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:18.204344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:18.206008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:18.220630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 473/947] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:18 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:18.251978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97652768-d463-4ae6-a0f7-6004713c3124 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:18.294360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:18.295187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:18.295564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:18.295930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:18.296777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:18.297241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:18.299277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0482e-d412-4579-925c-f5d36d93a46c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:18.305588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 475/948] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:18.311268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:18.388604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:18.445276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44795004-9e86-46fb-9f7e-d3cc67764cf2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:18.484689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:18.485365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:18.485654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:18.485933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:18.486719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:18.488138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1e1ac-c2a3-458b-8209-eb2394a68a76 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:18.511304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 474/949] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:19.808532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1edbb-95d0-49ca-8238-a51911c41996 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:19.808532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1edbb-95d0-49ca-8238-a51911c41996 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:19.808532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1edbb-95d0-49ca-8238-a51911c41996 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:19.809610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 476/950] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:19.816809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:19.817281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:19.817356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:19.835428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:19.837326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:19.945301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 475/951] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:19.976084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1edbb-95d0-49ca-8238-a51911c41996 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.053058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.144786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1edbb-95d0-49ca-8238-a51911c41996 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.189295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.189976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.190263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.190544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.191269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:20.192763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:20.194087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING keystone.server.flask.common [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 14:59:20.208188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.sql.core [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 14:59:20.208188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 14:59:20.208188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [parameters: {'user_id': '50db2f25c4034043a9c747ca37c3de78', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 14:59:20.208188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80315) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-6f2a2052-f807-4c37-8258-86b40f58046f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() May 27 14:59:20.216349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:59:20.217230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application yield resource May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application next(self.gen) May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.rollback() May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.force_reraise() May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise self.value May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application session.commit() May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 14:59:20.218168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._prepare_impl() May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.session.flush() May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._flush(objects) May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application compat.raise_( May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application flush_context.execute() May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rec.execute(self) May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 14:59:20.219313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 14:59:20.221164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result = self._query(query) May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application conn.query(q) May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application result.read() May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 14:59:20.223625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [parameters: {'user_id': '50db2f25c4034043a9c747ca37c3de78', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:20.225804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 14:59:20.232998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.236232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 477/952] 10.222.0.41 () {60 vars in 1131 bytes} [Fri May 27 14:59:19 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:20.241605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.260685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.263783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f9f7-462d-4d79-a16d-6853ea3c03f4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.264075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f9f7-462d-4d79-a16d-6853ea3c03f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.264397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112f9f7-462d-4d79-a16d-6853ea3c03f4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.272661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 478/953] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:20.275300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.275632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.275915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.295564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:20.297817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:20.345855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75861b27-2ff5-45ed-aecd-d43b832574aa None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.388499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 479/954] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:20.388923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.388923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.388923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.389689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.400485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:20.400485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:20.400721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:20.418391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:20.419606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 14:59:20.420747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:20.427237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 476/955] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:20.428825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.436779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.523290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaa605-ad9f-42a9-8e2b-84c206191074 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.541586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.550218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac476-57c4-4cde-8ea3-3f6a05854deb None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.568500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.569418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.569685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.569983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.583607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:20.592271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:20.592929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:20.594562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:20.595420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:20.598181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.598695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.599176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.599681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 480/956] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:20.600542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.600816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:20.601346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:20.603022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fda22b-1905-460b-a06e-35ec84181790 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:20.645761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 477/957] 10.222.0.41 () {56 vars in 1101 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:20.701617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.720720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7adf-c2c3-4fa2-bf8e-d1b847652241 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.757765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc0a5-71db-4afd-8df3-4e03c610f020 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.758104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc0a5-71db-4afd-8df3-4e03c610f020 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.758335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc0a5-71db-4afd-8df3-4e03c610f020 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.760220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 478/958] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:20.760333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.761043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.761360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.761671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.762434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:20.762746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:20.764359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:20.766414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.766800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.767087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.775276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 481/959] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:20.781132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.785160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:20.786390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:20.866098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 479/960] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:20.890597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:20.905580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8503113-0160-42ab-8f7f-5a62f3e7c4fb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:20.960436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:20.960436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:20.960436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:20.960436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:20.974594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:20.976888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:20.986333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:20.996899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:20.997842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:20.998731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:20.998731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:20.998731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:20.998731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:21.003521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 482/961] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:21.004891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.010037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.036013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43f502-ebbb-46b9-b518-dbe82b26161b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.079962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.081046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.081479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.081931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.082749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.084276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:21.099646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:21.100919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.106979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 480/962] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:21.110492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.121964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9d363-181b-4d95-9e97-2ab842116621 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.160457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.161139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.161425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.161702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.162419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:21.162728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.164105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30634f1-f491-4dfc-bda9-2ad2625f6650 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.178808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 483/963] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:21.178808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce021-f022-4855-a69b-42cef62cef35 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.178808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce021-f022-4855-a69b-42cef62cef35 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.178808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce021-f022-4855-a69b-42cef62cef35 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.179670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 484/964] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:21.182191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce021-f022-4855-a69b-42cef62cef35 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.216189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66356-0474-4f83-9a91-ff375d218735 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.242301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.249432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.250627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.251071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.251361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.252055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:21.252364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.253756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f276d-a8a7-4600-8e7a-dca7eaf20682 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.258790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 481/965] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:21.259959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.260245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.260558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.276290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:21.277503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:21.280183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.287823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce021-f022-4855-a69b-42cef62cef35 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.322968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.323781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.324065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.324341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.331683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.333173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.344325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 482/966] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:21.345588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:21.349483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:21.350567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 14:59:21.351848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:21.354061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 485/967] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:21.359441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.456906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d5521-83be-4f48-862e-98b55727c4cc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.462835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6acb8b-cb26-4417-9ca8-62014a764c6a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.483321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.491324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.491990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.492304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.492622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.493333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:21.493641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.495019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.499104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.499773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.499861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 483/968] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:21.500124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.500412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.500649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.501268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:21.501583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.503524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.505719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.522003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 486/969] 10.222.0.41 () {56 vars in 1149 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:21.522861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.609904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bf5a3-2761-4827-9c2b-a2d2c0e95293 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.623119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbe0d9-43fa-4dac-abc6-64e3a41440ae None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.645394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.646221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.646504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.646788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.647446 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.648870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.657961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.658557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.658836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.659111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.659887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:21.660663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:21.661877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:21.662772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:21.663551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.663631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 484/970] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:21.663631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.692496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 487/971] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:21.692496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.732678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.743121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:21.789679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd81ca8-9749-43ed-81e5-e71029415a95 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.827304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c314b0-6a82-4b6f-a1fd-d8e2dd290dab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:21.833144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.833825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.834112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.834394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.870470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:21.871149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:21.871149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:21.871149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:21.871149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:21.873312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.874128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.879568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 488/972] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:21.885518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:21.928710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:21.930180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:21.993816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:22.002958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8dd13f-2049-4443-b596-d544eb363dcf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.003466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:22.039838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.040534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.041064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.041173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 485/973] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:22.041265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.041801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:22.042116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.043516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.046842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:22.056865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 489/974] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:22.161690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcde644-c98b-44af-b715-f0e135444b8b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.204690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.205781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.206063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.206388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.220854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.222345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:22.242603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:22.243592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:22.244644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:22.244644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:22.244644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:22.244644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:22.244644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 486/975] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:22.251840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:22.361999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c173b-f1e1-4837-8378-91ce4bd0a532 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.403350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.404010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.404291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.404597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.405315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:22.405619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.408017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.420413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 490/976] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:22.426088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:22.448822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:22.531269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106d627-7232-463d-8119-1dd838565af7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.568475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.569270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.569554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.569827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.576514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.577943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:22.592583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:22.593476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:22.595047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:22.596772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 487/977] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:22.600247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:22.619544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:22.705020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c020413d-7878-42ca-a5d7-ccf358c437bb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.740209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.740903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.741188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.741465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.742280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:22.742595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.744035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69682c51-85c3-413a-8a33-8ba0300626a8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.756897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 491/978] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:22.762841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:22.849409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:22.870809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212c12d-5d64-425f-a504-c97aa573c175 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:22.907396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:22.908098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:22.908384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:22.908707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:22.909605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] RBAC: Adding query filter params (role.id=44f51dd61a89463ea1de514abf8669a9, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:22.909946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:22.911462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aca27d-136d-448c-8089-1b14af29b61b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:22.922545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 488/979] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:22 2022] GET /identity/v3/role_assignments?user.id=fa4fbb1e67c24c2b8a1d816c59c75a1b&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=44f51dd61a89463ea1de514abf8669a9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:23.937795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357c9-edbe-4684-9cad-9fabc31d9614 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:23.938322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357c9-edbe-4684-9cad-9fabc31d9614 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:23.938322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a357c9-edbe-4684-9cad-9fabc31d9614 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:23.939712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 492/980] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:23.945174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:23.945570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:23.945737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:23.963236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:23.964427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:24.033132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 489/981] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:24.055914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a357c9-edbe-4684-9cad-9fabc31d9614 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.145505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:24.169369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a357c9-edbe-4684-9cad-9fabc31d9614 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.204778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.205510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.205794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.206071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.206720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.208105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-580aa8f4-bee7-4271-a36f-1233fcb8642e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:24.221312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:24.222322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:24.223169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:24.230814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 493/982] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:24.231044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.273696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdfe6e-f818-4ad5-9d46-52fa7e159dca None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.274016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdfe6e-f818-4ad5-9d46-52fa7e159dca None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.274268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdfe6e-f818-4ad5-9d46-52fa7e159dca None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.276484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 494/983] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:24.280683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.280968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.281768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.299161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:24.300392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:24.312604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:24.340761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f216-c8ab-413b-9473-3df62af12b97 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.372311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 495/984] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:24.383266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.384242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.385296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.385582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.386297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:24.386613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.387982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.396491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 490/985] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:24.396633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.398739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.472064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:24.506402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d64790-a326-469a-8525-cbfd33911c8a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.507568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd926a-206f-4435-a6ed-a415130e3e65 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.545223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.545436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.545897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.546069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.546187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.546374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.546473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.546690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.547789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:24.548078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.548285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.549551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.549681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.555266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 491/986] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:24.561302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.564571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 496/987] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:24 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:24.572545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:24.677197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef7f56-2f4b-42d3-84bb-e8aa444e6f84 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.722186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.722780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.723067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.723348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.724166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] RBAC: Adding query filter params (interface=admin, service_id=53d73fafabad49c7a9b7dd64a0489170, region_id=RegionOne) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:24.724515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.725921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17915771-d4b0-4175-a429-046bcf89a3e2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.731826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 492/988] 10.222.0.41 () {56 vars in 1249 bytes} [Fri May 27 14:59:24 2022] GET /identity/v3/endpoints?service_id=53d73fafabad49c7a9b7dd64a0489170&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:24.738978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:24.846687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a9790-b813-4a9c-a066-9b7a7e380e54 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:24.876307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:24.883190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:24.884266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:24.884955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:24.885235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:24.885921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:24.887446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059395a-2173-49aa-9572-e59bb13b042c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:24.894705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 497/989] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:24 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:25.174869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340538e2-b86a-442b-a593-c4f29bad8892 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:25.175387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340538e2-b86a-442b-a593-c4f29bad8892 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:25.175492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340538e2-b86a-442b-a593-c4f29bad8892 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:25.177042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 493/990] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:25.182011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:25.182422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:25.182585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:25.199582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:25.200797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:25.270871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 498/991] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:25.292880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340538e2-b86a-442b-a593-c4f29bad8892 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:25.357348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:25.408124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340538e2-b86a-442b-a593-c4f29bad8892 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:25.442561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:25.443284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:25.443567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:25.443844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] PATH_INFO: `/v3/services/image` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:25.444532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:25.445908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:25.458233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:25.459419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:25.460284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 494/992] 10.222.0.41 () {56 vars in 1105 bytes} [Fri May 27 14:59:25 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:25.466116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:25.568981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257f00-3c81-4289-95ae-113e5b8bea4e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:25.597060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:25.606991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:25.607659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:25.607946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:25.608225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:25.608960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] RBAC: Adding query filter params (name=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:25.609332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:25.610720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7cfb0a-f2d2-4903-b497-9e7cdebcc045 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:25.620151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 499/993] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:25.621430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:25.725785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d37655-d47a-419d-88dd-fc32bf59d22a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:25.754131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:25.764145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:25.764855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:25.765141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:25.765470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:25.766755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] RBAC: Adding query filter params (type=image) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:25.767069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:25.768436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767b316-8834-4de2-a501-8a17b8de2680 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:25.774055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 495/994] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:26.865555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a754f8-874b-449a-aa2e-3520218c8410 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:26.866088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a754f8-874b-449a-aa2e-3520218c8410 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:26.866088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a754f8-874b-449a-aa2e-3520218c8410 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:26.867725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 500/995] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:26.873044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ce5a-3727-4466-a539-4bb2d319f788 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:26.873475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ce5a-3727-4466-a539-4bb2d319f788 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:26.873654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248ce5a-3727-4466-a539-4bb2d319f788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:26.891999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a248ce5a-3727-4466-a539-4bb2d319f788 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:26.893197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a248ce5a-3727-4466-a539-4bb2d319f788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:26.969188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 496/996] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:26.991884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a754f8-874b-449a-aa2e-3520218c8410 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.098092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a754f8-874b-449a-aa2e-3520218c8410 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.129239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245f6a-6fb1-48b8-a38a-e157ecc9b484 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.129450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245f6a-6fb1-48b8-a38a-e157ecc9b484 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.129726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245f6a-6fb1-48b8-a38a-e157ecc9b484 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.131225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 497/997] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:27.133897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.134971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.135148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.135401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.136058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.136154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.136399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.136732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.137484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:27.150349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:27.151460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:27.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 501/998] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:27.158047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:27.158183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.159236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:27.217337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.236487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 498/999] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:27.257120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.267290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10745927-55f8-4057-8a42-357a10267d08 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.303385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.304045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.304628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.304628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.305319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:27.305624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.307000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.312249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 502/1000] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:27.317564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.317690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.373588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d702742-4e02-4898-92e4-002f1c0bd7cb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.416634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.417445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.417838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.418116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.418887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.420261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.427835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1d3f4-a135-4139-8184-5ee59b5bb55c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:27.433583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:27.434547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:27.435418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 499/1001] 10.222.0.41 () {56 vars in 1126 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:27.441093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.447682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.468038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.469296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.469589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.469869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.470678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:27.471075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.472486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.477783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 503/1002] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:27.483329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.553037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a959a5-c676-42b0-9bca-d745e8e2c27c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.568226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.590971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.591703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.592066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.592355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.592776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a693368-dd6d-40d6-9fa4-da08c7adac2c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.593085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:27.593435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.595596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.600909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 500/1003] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:27.606277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.632394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.633090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.633378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.633663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.634454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] RBAC: Adding query filter params (interface=public, service_id=aff68dbe8bfa43ce92c2e1bd18a7d2d1, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:27.634830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.636175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.648631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 504/1004] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/endpoints?service_id=aff68dbe8bfa43ce92c2e1bd18a7d2d1&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:27.649534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.699444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.716532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92363423-3bf9-482f-9d18-c6b64b7a40b5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.754017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.754683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.754969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.755249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.755951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:27.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.757679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.760687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf76f61-2bcd-4b47-aa62-85af9eeaafc9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.762572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 501/1005] 10.222.0.41 () {56 vars in 1122 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:27.767916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:27.799801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.800522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.800808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.801084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.801790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.803225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa9984-fc43-4aa7-a61a-b5d01105af4e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.811311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 505/1006] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:27 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:27.821530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:27.889370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc3bbe-3277-4933-95d0-c8518bc13109 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:27.930211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:27.930887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:27.931174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:27.931456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:27.932217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:27.933648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e24b2e-a352-4fa8-9a23-1d7560f088b4 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:27.956487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 502/1007] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:28.007858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817776a-eeb7-4fdf-a14c-aa25f909a23e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:28.008503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817776a-eeb7-4fdf-a14c-aa25f909a23e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:28.008503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817776a-eeb7-4fdf-a14c-aa25f909a23e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:28.010203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 506/1008] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:28.017276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0412be-4a07-40ec-af33-df2565b4f84e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:28.017712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0412be-4a07-40ec-af33-df2565b4f84e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:28.018098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0412be-4a07-40ec-af33-df2565b4f84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:28.036285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9a0412be-4a07-40ec-af33-df2565b4f84e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:28.037565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9a0412be-4a07-40ec-af33-df2565b4f84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:28.121580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 503/1009] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:28 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:28.144982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817776a-eeb7-4fdf-a14c-aa25f909a23e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:28.253427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817776a-eeb7-4fdf-a14c-aa25f909a23e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:28.264496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:28.290528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:28.291283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:28.291638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:28.291984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:28.292817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:28.294340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d36f8-f824-4054-b33b-d0eb72d29314 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:28.316476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 507/1010] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:28 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:30.092694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5335a-fa05-4f4b-ab6d-1db24fa08adb None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.093219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5335a-fa05-4f4b-ab6d-1db24fa08adb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.093336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5335a-fa05-4f4b-ab6d-1db24fa08adb None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.095207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 504/1011] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:30.100743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.101205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.101291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.119652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:30.120882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:30.180276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f979e-10e4-4cab-b604-a9b9dad9ccf0 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.180609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f979e-10e4-4cab-b604-a9b9dad9ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.180870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f979e-10e4-4cab-b604-a9b9dad9ccf0 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.182375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 505/1012] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:30.189011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.189824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.190687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.200495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 508/1013] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:30 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:30.211567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:30.213018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:30.223839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.242412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.291597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 506/1014] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:30 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:30.316153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.333659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.343162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddab577-3278-4f98-9bec-1ba8d83bc58f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.388228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.389074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.389349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.389629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.390485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.392206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:30.404582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:30.405599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:30.407593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 509/1015] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:30.412411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.444240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ab228-43be-4f3c-a2bd-08d2a029b7e6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.480975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.482249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.482535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.482919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.484640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.486435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.492896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 507/1016] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:30.493903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.519338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a241ff-f288-46b3-8803-62ed8a38db11 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.523013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.555792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.556467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.556778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.557058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.557765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:30.558075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.559453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df6670-df50-45d3-9d02-530173b54421 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.564354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 510/1017] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:30.565579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b87dd-90e7-4d5d-86eb-460cb25ae97d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.565867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b87dd-90e7-4d5d-86eb-460cb25ae97d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.566152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b87dd-90e7-4d5d-86eb-460cb25ae97d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.567366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 511/1018] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:30.572001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b87dd-90e7-4d5d-86eb-460cb25ae97d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.598959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.604794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d74efe-5185-41e3-bce1-2fbabd2b0691 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.639079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.639746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.640038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.640863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.641577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.643014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e4a5b-dbdd-46b5-a652-9c094faf16c8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.668270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 508/1019] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:30 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:30.669554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.669831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.670114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.679009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b87dd-90e7-4d5d-86eb-460cb25ae97d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.695940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:30.697224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:30.723383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.724078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.724392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.724677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.725434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:30.725749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.727204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.732987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 512/1020] 10.222.0.41 () {56 vars in 1112 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:30.739055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.780536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 509/1021] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:30 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:30.804789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:30.815708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.839402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:30.857259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d1cba-0f98-4625-a39f-98fb9170a249 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.898752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.899527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.899817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.900820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.900930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.902455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.922929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031429-dd6c-45fc-9d18-7e573b01e2ee None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:30.924906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 513/1022] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:30.973394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:30.974206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:30.974492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:30.974770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] PATH_INFO: `/v3/services/image` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:30.975433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:30.976858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:30.989453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:30.990585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:30.991413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 510/1023] 10.222.0.41 () {56 vars in 1105 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:30.997090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:31.111961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8e4e5-e968-421a-b53b-b675d7fbffe4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:31.157965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:31.158735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:31.159249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:31.159696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:31.160974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] RBAC: Adding query filter params (name=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:31.161511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:31.163605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:31.170485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 514/1024] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:31.178602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:31.274437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:31.308975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3e084-949a-4e6f-bdf4-c608d5104982 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:31.345069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:31.345744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:31.346026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:31.346303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:31.347012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:31.347374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:31.348775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:31.353677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 511/1025] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:31.359581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:31.371343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:31.466850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964c128-a044-48aa-bf38-8d8a080015aa None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:31.500980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:31.501640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:31.501930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:31.502207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:31.502991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] RBAC: Adding query filter params (interface=public, service_id=3b24b570e3f645c88ac0ace34d13a76b, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:31.503303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:31.504709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ff67-1d54-4de3-a4b4-d3e2d4391e92 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:31.514409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 515/1026] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 14:59:31 2022] GET /identity/v3/endpoints?service_id=3b24b570e3f645c88ac0ace34d13a76b&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:31.516129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:31.629511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ec72f-ed75-4912-8a53-c0dcf9d1c1d5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:31.659623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:31.666086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:31.666755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:31.667042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:31.667320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:31.667996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:31.669431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bfca0-f7b0-4be8-9e47-e82a513b0e33 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:31.684537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 512/1027] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 14:59:31 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 14:59:33.071879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69101864-0a90-4468-8c47-9c0d7697f7ce None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.072404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69101864-0a90-4468-8c47-9c0d7697f7ce None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.072404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69101864-0a90-4468-8c47-9c0d7697f7ce None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.076485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 516/1028] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:33.079700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed9f6-e524-4031-bec3-4d879762df4e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.080115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed9f6-e524-4031-bec3-4d879762df4e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.080289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ed9f6-e524-4031-bec3-4d879762df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.098447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-648ed9f6-e524-4031-bec3-4d879762df4e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:33.099673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-648ed9f6-e524-4031-bec3-4d879762df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:33.172358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 513/1029] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:33.197876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69101864-0a90-4468-8c47-9c0d7697f7ce None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.272823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:33.305610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69101864-0a90-4468-8c47-9c0d7697f7ce None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:33.306584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b9a5f-ca4b-4273-bb99-3049a015bb16 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.306879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b9a5f-ca4b-4273-bb99-3049a015bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.307162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b9a5f-ca4b-4273-bb99-3049a015bb16 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.308582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 514/1030] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:33.314156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.314430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.314709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.332080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:33.333230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:33.344570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:33.345370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.345655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.345935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.347163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:33.348653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:33.350432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 517/1031] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:33.355995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.411769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 515/1032] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:33.435443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.469332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a701db-d692-4d18-ba7b-55303383db59 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:33.499089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:33.509242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:33.509914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.510197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.510474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.511192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:33.512654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:33.523389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:33.536493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 518/1033] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:59:33 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 14:59:33.552568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc8ad1-5750-4fdb-a065-4b0600aa7813 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:33.591675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:33.592434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.592744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.593016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.606254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:33.615239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:33.616277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:33.617513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 516/1034] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:33.623495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.740649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b4e55-2cd4-4114-8981-4c5c134c82ea None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:33.780435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:33.781293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.781789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.782076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.782828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:33.783282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:33.785060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad39f8-993f-4995-bdf6-6ec59a39e78e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:33.794507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 519/1035] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:33.800722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.901987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216740d3-0b56-45f0-b3ac-e04839340abc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.902180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216740d3-0b56-45f0-b3ac-e04839340abc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.902473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216740d3-0b56-45f0-b3ac-e04839340abc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.903562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551eaa7-5020-4404-b9d4-dad37c8cb9b8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:33.904018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 520/1036] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:33.909639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f7fa2-821a-4169-a898-506d9485886c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.909931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f7fa2-821a-4169-a898-506d9485886c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.910210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f7fa2-821a-4169-a898-506d9485886c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.922004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:33.928329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-427f7fa2-821a-4169-a898-506d9485886c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:33.929627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-427f7fa2-821a-4169-a898-506d9485886c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:33.941367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:33.942160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:33.942436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:33.942706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:33.954039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:33.955485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:33.974583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:33.975498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:33.976381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:33.976381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:33.976381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:33.976381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:33.976381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 517/1037] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:33.981285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:33.991666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.007438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 521/1038] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:34.030113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f7fa2-821a-4169-a898-506d9485886c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.097819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3da702-1018-4f92-9398-a2d09a2ddca3 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.138143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.138797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.139084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.139363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.140133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:34.140429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.141842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.148050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f7fa2-821a-4169-a898-506d9485886c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.153984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 518/1039] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:34.160198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.189046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.191706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.192530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.192984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.193108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] PATH_INFO: `/v3/services/image` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.193808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.195132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:34.207361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:34.208395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:34.209329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 522/1040] 10.222.0.41 () {56 vars in 1105 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:34.214690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.269587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57462e-02e4-47ca-999f-f9bf853dd393 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.273246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.307257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.308049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.308335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.308641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.316292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.317827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.326207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424b146-2a1c-45fa-bbd0-ba5d1e37a12e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:34.332976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:34.333952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:34.335021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 519/1041] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:34.339767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.367935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.368741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.369026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.369454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.370035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:34.370349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.371738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.383350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 523/1042] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:34.384279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.441292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.455343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288aee8-86f6-49e6-b123-f781f4e33244 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.493366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.493974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.494260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.494686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.495361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:34.495674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.497134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.497751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8670d3-16fb-47ee-ac0c-d5dee22e1cc9 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.515841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.516883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 520/1043] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:34.522618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.540317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.541024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.541315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.541595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.542299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:34.542608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.544016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.549062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 524/1044] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:34.554638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.628736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba85645-fcb7-4732-b71b-f07469a741f0 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.663543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f8416-193f-4381-a33f-7bcf3f5d86a1 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:34.663895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.664589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.664877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.665152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.682039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.699610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:34.700312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:34.700616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:34.700897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:34.701624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.704026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.736489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 525/1045] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 14:59:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:34.754744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:34.756466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:34.875744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 521/1046] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:34.877140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:34.916504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:34.996538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e09fa-3e9a-4e45-bc06-7e35c9e759d4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:35.031058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:35.031887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.032170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.032476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.043055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:35.044580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:35.062852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:35.063861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:35.064805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:35.064805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:35.064805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:35.064805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:35.064805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 526/1047] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:35.069797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:35.119190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:35.174189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edfc18-93da-4e86-ad7f-e09684910e43 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:35.208686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:35.209346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.209626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.209897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.210611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:35.210917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:35.212273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:35.228481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 522/1048] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:35.230356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:35.320071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:35.337488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3c5af-3d2a-4d3a-ac87-17aeafc8cf52 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:35.371764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:35.376482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.376482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.376482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.385614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:35.387103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:35.402257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:35.403171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:35.404058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:35.405079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 527/1049] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:35.410180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:35.517395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ece61-b605-48eb-9ca0-706137ec5153 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:35.524688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:35.553251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:35.553918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.554201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.554478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.555256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:35.555566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:35.557036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6916c-d66c-4e0d-b1b1-f023c8087aef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:35.574628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 523/1050] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:35.575971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:35.675905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8aaf13-5a79-4c73-9a31-343692828161 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:35.714413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:35.715338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.715622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.715898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.716759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] RBAC: Adding query filter params (role.id=44f51dd61a89463ea1de514abf8669a9, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:35.717074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:35.719142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba41b5-1281-4fbb-b310-4b55cf0ff3d7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:35.732489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 528/1051] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:35 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=44f51dd61a89463ea1de514abf8669a9 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:35.883099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184a3a-4655-4a5f-aa33-733fb472f76d None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.883442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184a3a-4655-4a5f-aa33-733fb472f76d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.883619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184a3a-4655-4a5f-aa33-733fb472f76d None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.885319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 524/1052] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:35.892315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:35.892693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:35.892928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:35.911276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:35.912509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:35.914607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:35.988672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 529/1053] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:36.007598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184a3a-4655-4a5f-aa33-733fb472f76d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:36.120520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184a3a-4655-4a5f-aa33-733fb472f76d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:36.156179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:36.156905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:36.157192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:36.157471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:36.167825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:36.175601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:36.177084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:36.177996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:36.180544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 525/1054] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:36.183456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:36.285855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faee9b-4b04-46f4-898d-c6394869d8ca None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:36.320100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:36.320793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:36.321078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:36.321353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:36.322098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:36.322406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:36.323783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:36.329941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:36.338715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 530/1055] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:36.340391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:36.370195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:36.455076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059c49-8b3f-4509-936e-ba1d160fbc08 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:36.490859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:36.491740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:36.492030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:36.492307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] PATH_INFO: `/v3/users/e414c28c3f6546f7b889121b35c1aa34` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:36.506629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] RBAC: Authorizing `identity:get_user(user_id=e414c28c3f6546f7b889121b35c1aa34)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:36.508142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:36.514521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 526/1056] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/users/e414c28c3f6546f7b889121b35c1aa34 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:36.516285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:36.619699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8505b-4ad3-4afa-9363-203b0f0767b4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:36.653403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:36.654044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:36.654333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:36.654612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:36.721558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:36.724683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:36.726184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:36.829255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 531/1057] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:36.835388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:36.925230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:36.943026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a3c8b-5797-4880-8290-08fb0a721048 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:36.978392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d5222-800e-4118-b55d-598245e08607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:36.979178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d5222-800e-4118-b55d-598245e08607 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:36.979468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d5222-800e-4118-b55d-598245e08607 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:36.979746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d5222-800e-4118-b55d-598245e08607 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:36.980954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d5222-800e-4118-b55d-598245e08607 None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:36.982392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d5222-800e-4118-b55d-598245e08607 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:36.988547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 527/1058] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:36.993071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:37.029318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bff-8db5-4192-a8b8-4022ac283ca9 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.029635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bff-8db5-4192-a8b8-4022ac283ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.029885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b84bff-8db5-4192-a8b8-4022ac283ca9 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.032409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 528/1059] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:37.037457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.037933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.038212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.055255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:37.056419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:37.104594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:37.108231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36190c-07bf-45f6-bd01-1a046db94f4b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:37.129033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 529/1060] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:37.145104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:37.145754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.146034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.146308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.147124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user.id=e414c28c3f6546f7b889121b35c1aa34) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:37.147433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:37.148892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:37.150750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:37.159938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 532/1061] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:36 2022] GET /identity/v3/role_assignments?user.id=e414c28c3f6546f7b889121b35c1aa34&scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:37.243136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:37.273483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802f57f-d3e5-4ae0-843c-15c274aedbc3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:37.310924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:37.311597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.311933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.312263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.399498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:37.400991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:37.486013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:37.506530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 530/1062] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:37.512172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:37.616387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46f82e-9c56-4afe-99c6-70329bd535e3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:37.651637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:37.652430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.652738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.653013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.663890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:37.665366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-fde38034-b8fd-46b8-b933-250791a526d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:37.682501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:37.683460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:37.684380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:37.684380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:37.684380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:37.684380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:37.689354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 533/1063] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:37.690706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:37.797474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd91844-74b0-4a8e-812f-8ba6ded9a2e6 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:37.834162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:37.834830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.835117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.835398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.836125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:37.836433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:37.837840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13b2f0-2eba-48f7-aa36-35ff1ff0103d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:37.856477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 531/1064] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:37.977699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ba89-8147-4fb9-9300-9aa9b7653f4b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.978071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ba89-8147-4fb9-9300-9aa9b7653f4b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.978282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860ba89-8147-4fb9-9300-9aa9b7653f4b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:37.979870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 534/1065] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:37.986461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:37.986790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:37.987036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.005758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:38.006968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:38.045858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:38.080855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 532/1066] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:38.105292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ba89-8147-4fb9-9300-9aa9b7653f4b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:38.125519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:38.212919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860ba89-8147-4fb9-9300-9aa9b7653f4b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:38.246901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:38.247704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:38.247983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:38.248254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.258216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:38.267293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:38.268489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:38.269935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 535/1067] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:38.275136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:38.380222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7541031-70e1-4299-aba8-feb12bcf47ba None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:38.415091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:38.415760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:38.416039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:38.416310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.417080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:38.417389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:38.418752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:38.426797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 533/1068] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:38.432658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:38.509102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:38.532086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1609997-2b24-4f23-a740-5d1dc643e078 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:38.564680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:38.565465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:38.565750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:38.566024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.576989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:38.578996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:38.595963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:38.597038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:38.597959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:38.597959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:38.597959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:38.597959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:38.597959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 536/1069] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:38.603158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:38.637454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:38.703469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abd05-c4ab-406a-be1c-f642c65b7f8a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:38.737683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:38.738346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:38.738630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:38.738908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.739623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:38.739931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:38.741331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:38.753096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 534/1070] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:38.758724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:38.858326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f99d1-dda0-45d5-bce9-b8c8ed342211 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:38.883867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:38.892608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:38.893341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:38.893623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:38.893897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:38.901612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:38.903063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:38.917604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:38.918741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:38.919750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 537/1071] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:38.924792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.021587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.028784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667fb9a-047d-4eb7-8e0f-d5cd6a566966 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.062292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.062987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.063270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.063544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.064359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:39.064693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.066107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.083275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 535/1072] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:59:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:39.088858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.189080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111c47-4d63-4a6f-9382-a76b7c6c5945 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.232056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.233019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.233307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.233588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.244492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.245953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f5d9c5f8-2d97-4a8f-bfc7-73d86b8cd137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:39.263330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:39.264320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:39.265893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:39.265893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:39.265893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:39.265893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:39.265893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 538/1073] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:39.271028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.376009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d754017-f4e8-4027-92f9-3ecc0c3475a8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.392348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5320fb-bd74-47ca-9922-9b2a160076d8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.392673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5320fb-bd74-47ca-9922-9b2a160076d8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.392959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5320fb-bd74-47ca-9922-9b2a160076d8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.394400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 539/1074] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:39.399301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.399593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.399877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.414180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.414845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.415132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.415410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.416140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:39.416476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.416579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:39.417700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:39.417859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.429605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 536/1075] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:39.435088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.449952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.453857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.489778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 540/1076] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:39.512316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.538268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ba1a-63fc-475a-992e-f4fdd97b23ef None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.570867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.571661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.572199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.572199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.579705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.581188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:39.597275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:39.598299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:39.599191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 537/1077] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:39.604071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.613322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c497-f00b-4eaf-b79a-a63c16c965cc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.648259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.649080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.649367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.649647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.660501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:39.668125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:39.669129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:39.670005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 541/1078] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:39.675246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.676542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.705979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.707038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce429-a0d7-48cb-81af-3a95cd52f916 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.742160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.742814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.743097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.743371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.744148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:39.744477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.745900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.757793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 538/1079] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:39.763411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.775958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1324c-846e-4756-bedb-724d4817c295 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.809627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.810285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.810565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.810838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.811574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:39.812017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.813547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.821618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 542/1080] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:39.828067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:39.862500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d397-4ba8-4d87-8d06-d5bb43c5e109 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.892741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.898228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.899163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.899455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.899734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/44f51dd61a89463ea1de514abf8669a9` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.901996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=f2d44da5ce124bd5b18462136f91fcb6, role_id=44f51dd61a89463ea1de514abf8669a9)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.903442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.916510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 539/1081] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:39 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/f2d44da5ce124bd5b18462136f91fcb6/roles/44f51dd61a89463ea1de514abf8669a9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 14:59:39.921767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:39.931886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54848-e1c3-47c1-95bb-fa850f0b8a34 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:39.970969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:39.971777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:39.972058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:39.972333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] PATH_INFO: `/v3/users/e414c28c3f6546f7b889121b35c1aa34` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:39.973659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] RBAC: Authorizing `identity:get_user(user_id=e414c28c3f6546f7b889121b35c1aa34)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:39.975140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:39.977085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 543/1082] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/users/e414c28c3f6546f7b889121b35c1aa34 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:39.983114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:40.092483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957063e5-fccc-4d96-ba87-5531c43bf472 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:40.129873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:40.130743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:40.131027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:40.131308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:40.132603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:40.134087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb41995-78c2-41df-af37-32300b5b0e55 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:40.135811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 540/1083] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:39 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:40.141830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:40.242692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f539-63e4-4061-8bfa-ecb287d0a646 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:40.275890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:40.276807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:40.277089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:40.277365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972/users/e414c28c3f6546f7b889121b35c1aa34/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:40.279324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user_id=e414c28c3f6546f7b889121b35c1aa34, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:40.281504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b902b5-35ed-4f7f-823b-bfae32c57a18 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:40.294405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 544/1084] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:40 2022] PUT /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972/users/e414c28c3f6546f7b889121b35c1aa34/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:42.116315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8daf-9cc1-45de-8759-87023d336ba4 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.116897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8daf-9cc1-45de-8759-87023d336ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.116988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb8daf-9cc1-45de-8759-87023d336ba4 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.118362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 541/1085] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:42.123638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.124049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.124342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.141216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:42.142384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:42.157210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.211040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 545/1086] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:42.235517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8daf-9cc1-45de-8759-87023d336ba4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.315760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.361048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb8daf-9cc1-45de-8759-87023d336ba4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.400268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.401112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.401393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.401667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.412058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-c33cd8da-2f2d-4f4b-9fe8-e3d286044293 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:42.420126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:42.421140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:42.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 542/1087] 10.222.0.41 () {56 vars in 1120 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:42.427326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.497929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e43-6b03-41ea-9296-d4226a000419 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.498219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e43-6b03-41ea-9296-d4226a000419 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.498499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d60e43-6b03-41ea-9296-d4226a000419 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.500703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 543/1088] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:42.505599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.505890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.506174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.523997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:42.525220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:42.528337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c832e9-d0fc-44f0-bc22-3d85c5679b0e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.563687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.564357 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.564670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.564949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.565688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:42.565996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:42.567377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:42.575314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 546/1089] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:42.581483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.592128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 544/1090] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 14:59:42.597199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.616171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.667975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.683076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc87a1-fee3-41ed-992e-b7f2278ebe8f None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.717193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.718672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.718957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.719096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be7bcd-52cc-463f-a543-b70381b38448 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.719256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.730588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:42.732097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:42.749571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:42.750605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:42.751496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:42.751496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:42.751496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:42.751496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:42.751496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 547/1091] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:42.756350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.757203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.758025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.758327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.758607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.769886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:42.777691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:42.778797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 14:59:42.779641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 545/1092] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:42.785324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.820639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.860522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647314d-976d-47da-ac24-024f1a72b058 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.885958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c2dc-b8b1-4f89-b5ac-82161a57dd73 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:42.894805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.895473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.895756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.896030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.896792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:42.897159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:42.897276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:42.898569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:42.910424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 548/1093] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:42.915858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:42.921910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:42.922520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:42.922808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:42.923113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:42.923866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:42.924176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:42.925581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:42.933825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 546/1094] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:42.940154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.014222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72dd2-bc8d-48d3-88a3-d9827cff6755 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.038790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.041524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a42777-6d12-488d-897b-afe719f6444a None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.050450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.051242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.051524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.051801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.059571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.061151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.075405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.076277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 14:59:43.076386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:43.077581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 14:59:43.078679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:43.079348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.079348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] PATH_INFO: `/v3/users/e414c28c3f6546f7b889121b35c1aa34` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.079514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 549/1095] 10.222.0.41 () {56 vars in 1093 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:43.079596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=e414c28c3f6546f7b889121b35c1aa34)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.080089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.081893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 547/1096] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:42 2022] GET /identity/v3/users/e414c28c3f6546f7b889121b35c1aa34 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:43.083539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.087126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.129588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.187007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e6bcc-4c89-40c3-9b7e-7c6cd4fa12bf None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.189116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7704191-ab49-4c30-974d-ce0a382dbcb1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.221167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.221836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.222121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.222476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.223309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:43.223622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.224273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.224942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.225104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.225216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.225499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.243116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 550/1097] 10.222.0.41 () {56 vars in 1133 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:43.248706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.292138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.297509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.298967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.342296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.354830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498df5-d1b8-45f3-81e5-ef419cb9a8e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.392993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.393652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.393964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.394249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.405529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 548/1098] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:43.411266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.466061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.467437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.509997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.511101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7c8b-a4d1-4b2f-9d7b-a55d7c42355f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.547883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.548781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.549061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.549336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.550509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.551942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.553672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 549/1099] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:43.559541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.569702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 551/1100] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 14:59:43.569862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.575810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.665230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248e188-a2ef-408d-8ed6-5dd75ee106de None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.677340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17ecee-c611-42d9-a6ca-eee5e49e33b7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.701567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.702261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.702741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.703021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.703912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user.id=e414c28c3f6546f7b889121b35c1aa34) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:43.704232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.705706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.711534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.711686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 550/1101] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/role_assignments?user.id=e414c28c3f6546f7b889121b35c1aa34&scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:43.712330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.712648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.712928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.725089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.726594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.728810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:43.748175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:43.749276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:43.750263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:43.750263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:43.750263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:43.750263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:43.750263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 552/1102] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:43.755752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:43.833943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:43.865782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184920a-6ab9-4965-b610-3cc8572140b5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:43.902251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:43.902914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:43.903195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:43.903531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:43.904255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:43.904595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:43.906328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:43.917767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 551/1103] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:43.923861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:44.029046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f14f3e-2d3f-468d-853d-d53692599f15 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:44.063806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:44.064669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:44.064981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:44.065277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:44.072068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:44.073651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 14:59:44.088009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:44.089100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 14:59:44.090450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 553/1104] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 14:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:44.095771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:44.195819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cbb66-7195-41f0-bf23-394686b839a7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:44.225392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:44.231046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:44.231703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:44.231983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:44.232256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:44.233153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:44.234052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:44.235795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cf837-6412-4df2-bbae-9f85c5864e09 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:44.248312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 552/1105] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 14:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:44.255249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:44.311204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:44.357918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e61360-1d1a-48c7-9ff2-851a6804c152 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:44.392800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:44.393410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:44.393698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:44.393980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:44.394805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] RBAC: Adding query filter params (role.id=44f51dd61a89463ea1de514abf8669a9, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=f2d44da5ce124bd5b18462136f91fcb6) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:44.395123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:44.396594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05bd63-2ae6-4503-93ec-f774b897ed37 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:44.402711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 554/1106] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:44 2022] GET /identity/v3/role_assignments?user.id=f2d44da5ce124bd5b18462136f91fcb6&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=44f51dd61a89463ea1de514abf8669a9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:45.997439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebdb5b-897c-41b8-a65f-04b083b65bcf None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:45.997976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebdb5b-897c-41b8-a65f-04b083b65bcf None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:45.997976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebdb5b-897c-41b8-a65f-04b083b65bcf None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:46.000270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 553/1107] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:46.005451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:46.005872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:46.006019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:46.023212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:46.024425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:46.090296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 555/1108] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:46.116437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebdb5b-897c-41b8-a65f-04b083b65bcf None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:46.215169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebdb5b-897c-41b8-a65f-04b083b65bcf None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:46.248747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:46.249540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:46.249826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:46.250101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:46.251266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:46.252719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde0f22-ca5d-44ef-a162-48e06cfeca57 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:46.254311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 554/1109] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:46.259903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:46.308359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:46.358746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217972b-aedd-404d-b1ed-cb72b6a2dccd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:46.393038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:46.393704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:46.393985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:46.394261 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:46.394973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:46.396465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15813525-be13-41c1-bcf2-b511354f15b4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:46.417029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 556/1110] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 14:59:46 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:48.722401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0cb4-439e-4fe1-bc2a-6b53cb71c340 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:48.722926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0cb4-439e-4fe1-bc2a-6b53cb71c340 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:48.722926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0cb4-439e-4fe1-bc2a-6b53cb71c340 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:48.724236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 555/1111] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:48.729741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:48.730159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:48.730296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:48.747032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:48.748200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:48.812981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 557/1112] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:48.836376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd0cb4-439e-4fe1-bc2a-6b53cb71c340 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:48.896601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:48.936763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd0cb4-439e-4fe1-bc2a-6b53cb71c340 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:48.970168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:48.971004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:48.971286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:48.971561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] PATH_INFO: `/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:48.973101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] RBAC: Authorizing `identity:get_role(role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:48.974416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:48.975777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 556/1113] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:48 2022] GET /identity/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:48.981559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.014720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:49.080831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a157e5-7845-4730-989a-ba98da3c6056 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.116302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.117503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.117845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.118223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] PATH_INFO: `/v3/users/959dd94d6a214c25a8b68f6ea960c8d7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.131731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] RBAC: Authorizing `identity:get_user(user_id=959dd94d6a214c25a8b68f6ea960c8d7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.133208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.134875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 558/1114] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:48 2022] GET /identity/v3/users/959dd94d6a214c25a8b68f6ea960c8d7 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:49.140820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.239161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b05e7-2553-4241-8a90-bec147080347 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.264999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:49.273054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.273699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.273985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.274262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.340255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.341735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.438116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 557/1115] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:49.443986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.534071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:49.543304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda23d0-482c-49db-b7e0-1c70ade5007d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.577578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.578428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.578718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.578992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.580156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.581624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d078d-7d2b-4b7f-bb3b-9696ddf0e7f9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.583304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 559/1116] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:49 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:49.589341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.617731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:49.688122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda35ef3-667b-42f2-ad1b-b2dce9d42e84 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.721529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.722181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.722468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.723004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.723561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user.id=959dd94d6a214c25a8b68f6ea960c8d7) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:49.723877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.725344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998420b-3371-40b2-95d5-89697476d1c3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.730761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 558/1117] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:49 2022] GET /identity/v3/role_assignments?user.id=959dd94d6a214c25a8b68f6ea960c8d7&scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:51.910330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28f2a1-a490-4d71-ba77-a1c408d5f2a9 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:51.910861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28f2a1-a490-4d71-ba77-a1c408d5f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:51.910861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28f2a1-a490-4d71-ba77-a1c408d5f2a9 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:51.912262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 560/1118] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:51.917429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:51.917812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:51.917996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:51.937801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:51.939497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:52.020221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 559/1119] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:52.042568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28f2a1-a490-4d71-ba77-a1c408d5f2a9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.144512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28f2a1-a490-4d71-ba77-a1c408d5f2a9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.177281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.178067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.178348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.178621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] PATH_INFO: `/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.180060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] RBAC: Authorizing `identity:get_role(role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.181462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.182805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 561/1120] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:52 2022] GET /identity/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:52.189478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.213087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:52.289030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591000-25f3-4f26-ae3d-83a3f7cf75f5 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.323994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.324981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.325270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.325550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] PATH_INFO: `/v3/users/959dd94d6a214c25a8b68f6ea960c8d7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.326767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] RBAC: Authorizing `identity:get_user(user_id=959dd94d6a214c25a8b68f6ea960c8d7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.328225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.329913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 560/1121] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:52 2022] GET /identity/v3/users/959dd94d6a214c25a8b68f6ea960c8d7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:52.337558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.366902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:52.439457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6343035a-099d-4b2e-b3b8-75845edd5b40 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.473906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.474762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.475048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.475325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.476514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.477961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba87a8-f917-4fbf-81c6-127c132bcde3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.479650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 562/1122] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:52 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:52.485540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.573612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:52.583635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74537f-bc39-436c-bb2e-5a3cd4b3b956 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.619045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.619919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.620202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.620500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972/users/959dd94d6a214c25a8b68f6ea960c8d7/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.622676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user_id=959dd94d6a214c25a8b68f6ea960c8d7, role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.624274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824428f3-606a-4f7e-97f2-1f218dda20d1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.637633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 561/1123] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 14:59:52 2022] PUT /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972/users/959dd94d6a214c25a8b68f6ea960c8d7/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:54.822282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29486b-9d6f-47f1-87fa-74091d846334 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.822815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29486b-9d6f-47f1-87fa-74091d846334 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.822929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29486b-9d6f-47f1-87fa-74091d846334 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.824314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 563/1124] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:54.830389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.830829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.830950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.847780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:54.849003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:54.914399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 562/1125] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:54.937973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29486b-9d6f-47f1-87fa-74091d846334 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.030588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.037574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29486b-9d6f-47f1-87fa-74091d846334 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.070375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.071221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.071508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.071781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] PATH_INFO: `/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.073315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] RBAC: Authorizing `identity:get_role(role_id=6d0ac795e42d4c4b8cedfbb5b209e0bd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.074644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.075997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 564/1126] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:54 2022] GET /identity/v3/roles/6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:55.082314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.160502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.179970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6848b-8d5a-4728-b13d-aae5eedf872f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.213587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.214452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.214990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.215270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] PATH_INFO: `/v3/users/959dd94d6a214c25a8b68f6ea960c8d7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.216438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] RBAC: Authorizing `identity:get_user(user_id=959dd94d6a214c25a8b68f6ea960c8d7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.218078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.219696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 563/1127] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/users/959dd94d6a214c25a8b68f6ea960c8d7 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:55.225373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.336953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7a742-6146-4b62-92d7-895b5840701a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.371049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.371693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.371980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.372259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.402584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.439993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.441487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.537457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 565/1128] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 14:59:55.543302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.640766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0569cb-b72d-4346-98d5-43f84537a092 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.673834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.674678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.674959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.675235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] PATH_INFO: `/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.676394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] RBAC: Authorizing `identity:get_project(project_id=d6e8c478ec9f4d4a94a8eb99f8f2b972)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.677852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715857a-db6e-4233-a604-876d5322b244 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.679536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 564/1129] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/projects/d6e8c478ec9f4d4a94a8eb99f8f2b972 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:55.685470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.749830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.787245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4879ebc-1075-41de-9d1d-4e87c6cc9606 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.820725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.821365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.821647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.821925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.822740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] RBAC: Adding query filter params (role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd, scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972, user.id=959dd94d6a214c25a8b68f6ea960c8d7) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:55.823269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.824510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbc489-6bc3-4e2f-8e64-86a9e3d38621 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.830952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 566/1130] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/role_assignments?user.id=959dd94d6a214c25a8b68f6ea960c8d7&scope.project.id=d6e8c478ec9f4d4a94a8eb99f8f2b972&role.id=6d0ac795e42d4c4b8cedfbb5b209e0bd => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:58.067285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147135ed-d180-48cc-9c99-7c27fea47ad2 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.067817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147135ed-d180-48cc-9c99-7c27fea47ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.067817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147135ed-d180-48cc-9c99-7c27fea47ad2 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.069248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 565/1131] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 14:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:58.074553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2218-70b5-484a-af42-38dce6782854 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.074973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2218-70b5-484a-af42-38dce6782854 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.075127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f2218-70b5-484a-af42-38dce6782854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.092794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-916f2218-70b5-484a-af42-38dce6782854 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:58.094096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-916f2218-70b5-484a-af42-38dce6782854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:58.162062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 567/1132] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 14:59:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 14:59:58.184999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147135ed-d180-48cc-9c99-7c27fea47ad2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:58.204463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:58.287028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147135ed-d180-48cc-9c99-7c27fea47ad2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:58.320636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:58.321363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.321647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.321926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.323089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:58.324547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2210b-5a44-41c9-ac26-480d37a18c16 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:58.326159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 566/1133] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 14:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:58.332062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f2218-70b5-484a-af42-38dce6782854 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:58.391070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:58.432826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f2218-70b5-484a-af42-38dce6782854 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:58.468631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:58.469288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.469572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.469851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.470570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:58.472024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173943c-3fc4-4bf4-bb16-0adcae335ffa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:58.496975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 568/1134] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 14:59:58 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:00.722759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2778-cb7a-49e6-bb83-0bc9c67f1689 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.723264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2778-cb7a-49e6-bb83-0bc9c67f1689 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.723264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2778-cb7a-49e6-bb83-0bc9c67f1689 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.724645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 567/1135] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:00.729921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.730321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.730485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.747621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:00.749544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:00.816632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 569/1136] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:00.841208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2778-cb7a-49e6-bb83-0bc9c67f1689 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.926032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:00.943089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2778-cb7a-49e6-bb83-0bc9c67f1689 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.976677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.977399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.977684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.977964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.979134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.980600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaffda-5bd5-412f-85d8-b7b3d8e3734e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.982214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 568/1137] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:00:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:00.988751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:01.028465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:01.085758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c2d15-be67-4cfe-9adb-7f6362ce20e2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:01.120123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:01.120808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:01.121097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:01.121378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:01.122099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:01.123671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84088b12-2fbf-4fc7-a6c9-4c43999e86fa None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:01.142775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 570/1138] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:03.385691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fbe70-9176-4ee4-b952-4da71ece9c37 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.386098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fbe70-9176-4ee4-b952-4da71ece9c37 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.386251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fbe70-9176-4ee4-b952-4da71ece9c37 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.387634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 569/1139] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:03.393792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.394183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.394353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.410961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:03.412145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:03.477567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 571/1140] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:03.501436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fbe70-9176-4ee4-b952-4da71ece9c37 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:03.601416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fbe70-9176-4ee4-b952-4da71ece9c37 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.634985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.635731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.636020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.636303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.647872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:03.655605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:03.656738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:03.657783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 570/1141] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:03.663025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:03.690524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:03.765043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bce6de-13de-4f21-801b-af7ef349edf5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.798720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.799381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.799666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.799944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.800717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:03.801030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:03.802411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:03.810347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 572/1142] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:03.816389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:03.831595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:03.914144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d21abb-9aca-4dad-b5bd-09a63fff71e2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.947241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.948088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.948369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.948672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] PATH_INFO: `/v3/users/ddbcbc10131247e3bb03c358a1efae8d` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.961921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] RBAC: Authorizing `identity:get_user(user_id=ddbcbc10131247e3bb03c358a1efae8d)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:03.963355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:03.965286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 571/1143] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:03 2022] GET /identity/v3/users/ddbcbc10131247e3bb03c358a1efae8d => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:03.971184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:04.058385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:04.070506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4bd58-07a8-473d-8aed-2d6d3d7943a4 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:04.105096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:04.105745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:04.106029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:04.106308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:04.175482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:04.177032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:04.260351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:04.273845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 573/1144] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:00:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:00:04.280493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:04.349641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:04.382253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669fed1-67db-4826-9ec9-15ac8e433e3b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:04.415667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:04.416546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:04.416837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:04.417116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] PATH_INFO: `/v3/projects/12c074c848724ded8a6926c08dab2638` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:04.418295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] RBAC: Authorizing `identity:get_project(project_id=12c074c848724ded8a6926c08dab2638)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:04.419750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0206b7-4581-4d3e-baf7-8dd72a43f85a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:04.422010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 572/1145] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:04 2022] GET /identity/v3/projects/12c074c848724ded8a6926c08dab2638 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:04.428346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:04.527927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1dc0-ae48-4406-81ff-4138c7d35523 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:04.562970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:04.563614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:04.563896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:04.564172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:04.565020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=12c074c848724ded8a6926c08dab2638, user.id=ddbcbc10131247e3bb03c358a1efae8d) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:04.565334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:04.566772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91255f41-215a-4357-8d85-5789d6c4de45 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:04.572116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 574/1146] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:00:04 2022] GET /identity/v3/role_assignments?user.id=ddbcbc10131247e3bb03c358a1efae8d&scope.project.id=12c074c848724ded8a6926c08dab2638&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:06.755767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1088-4699-4faa-9956-3596027fe935 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:06.756268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1088-4699-4faa-9956-3596027fe935 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:06.756268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b1088-4699-4faa-9956-3596027fe935 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:06.757651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 573/1147] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:06.762829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:06.763249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:06.763386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:06.780957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:06.782161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:06.830560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:06.851056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 575/1148] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:06.874451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b1088-4699-4faa-9956-3596027fe935 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:06.974192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b1088-4699-4faa-9956-3596027fe935 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:07.007217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:07.007927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:07.008206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:07.008498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:07.015536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:07.019461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:07.027212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:07.028412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:07.030293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 574/1149] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:07.035380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:07.132316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c182970-4085-431a-a8e9-35488eaae926 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:07.165312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:07.165966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:07.166249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:07.166525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:07.167258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:07.167566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:07.168981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:07.177813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 576/1150] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:07.183745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:07.282404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd82366-4601-4c87-b23e-a60a9ad1c4a2 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:07.315290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:07.316152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:07.316438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:07.316752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] PATH_INFO: `/v3/users/ddbcbc10131247e3bb03c358a1efae8d` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:07.317941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] RBAC: Authorizing `identity:get_user(user_id=ddbcbc10131247e3bb03c358a1efae8d)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:07.319378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:07.321039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 575/1151] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:07 2022] GET /identity/v3/users/ddbcbc10131247e3bb03c358a1efae8d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:07.327260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:07.372628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:07.427543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873670c-f96c-435b-86ea-f3ce51afedeb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:07.460404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:07.461288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:07.461572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:07.461849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] PATH_INFO: `/v3/projects/12c074c848724ded8a6926c08dab2638` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:07.463015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] RBAC: Authorizing `identity:get_project(project_id=12c074c848724ded8a6926c08dab2638)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:07.464750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd4d23-c15c-4039-87f0-2346794cd5b3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:07.466434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 577/1152] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:07 2022] GET /identity/v3/projects/12c074c848724ded8a6926c08dab2638 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:07.472678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:07.536438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:07.570835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116de77-d69c-4e18-a763-03997fc26fec None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:07.606024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:07.608230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:07.608796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:07.609304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] PATH_INFO: `/v3/projects/12c074c848724ded8a6926c08dab2638/users/ddbcbc10131247e3bb03c358a1efae8d/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:07.612877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=12c074c848724ded8a6926c08dab2638, user_id=ddbcbc10131247e3bb03c358a1efae8d, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:07.615661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21124275-89f7-4c81-aba0-a5db43ea2bf2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:07.631454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 576/1153] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 15:00:07 2022] PUT /identity/v3/projects/12c074c848724ded8a6926c08dab2638/users/ddbcbc10131247e3bb03c358a1efae8d/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:09.817784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af3b1-86f0-4363-8790-5b9f3312fc7b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.818315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af3b1-86f0-4363-8790-5b9f3312fc7b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.818438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af3b1-86f0-4363-8790-5b9f3312fc7b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.819705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 578/1154] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:09.824998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.825422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.825552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.843878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:09.845068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:09.910520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 577/1155] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:00:09.933670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4af3b1-86f0-4363-8790-5b9f3312fc7b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.034252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4af3b1-86f0-4363-8790-5b9f3312fc7b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.048096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.069135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.069854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.070138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.070415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.081215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:10.088792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:10.090154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:00:10.091202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 579/1156] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:10.096937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.152931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.197222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07946309-dd92-454d-ad49-d78e94a08d66 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.230961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.231613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.231893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.232171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.233924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:10.234235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.235592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.243413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 578/1157] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.250422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.349163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19eb97-6835-45f4-8254-4cf3b1ec50da None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.382434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.383308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.383595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.383872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] PATH_INFO: `/v3/users/ddbcbc10131247e3bb03c358a1efae8d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.385088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] RBAC: Authorizing `identity:get_user(user_id=ddbcbc10131247e3bb03c358a1efae8d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.386541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.388176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 580/1158] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/users/ddbcbc10131247e3bb03c358a1efae8d => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.394216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.491866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413d371-522b-4ba9-b1c4-89729c62c11c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.508021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.524901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.525539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.525825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.526107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.591450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.593569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.689942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 579/1159] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:00:10.695757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.738474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.794976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2990e-0b88-40da-b261-cb96435c250c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.827564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.828426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.828736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.829010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] PATH_INFO: `/v3/projects/12c074c848724ded8a6926c08dab2638` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.830179 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] RBAC: Authorizing `identity:get_project(project_id=12c074c848724ded8a6926c08dab2638)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.831609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddad49-019c-4326-b6d2-f05a319ede6c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.833317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 581/1160] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/projects/12c074c848724ded8a6926c08dab2638 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.839733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.858762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.936315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd76f97-d71d-4ccf-a5f6-d174499b9bb1 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.970891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.971476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.971761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.972040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.972897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=12c074c848724ded8a6926c08dab2638, user.id=ddbcbc10131247e3bb03c358a1efae8d) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:10.973214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.974642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f4f8e-197f-4695-aa94-5ade27c11c76 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.980347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 580/1161] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/role_assignments?user.id=ddbcbc10131247e3bb03c358a1efae8d&scope.project.id=12c074c848724ded8a6926c08dab2638&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:13.188990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc218-41d9-4cad-9bc9-c8ec1fbe7c03 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:13.189548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc218-41d9-4cad-9bc9-c8ec1fbe7c03 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:13.189548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc218-41d9-4cad-9bc9-c8ec1fbe7c03 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:13.190929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 582/1162] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:13.196188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:13.196610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:13.196773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:13.213739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:13.215163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:13.282392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 581/1163] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:13.304385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc218-41d9-4cad-9bc9-c8ec1fbe7c03 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:13.405585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc218-41d9-4cad-9bc9-c8ec1fbe7c03 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:13.414558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:13.439577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:13.440155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:13.440437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:13.440739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:13.506860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:13.508299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:13.603074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 583/1164] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:00:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:00:13.609945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:13.624727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:13.717978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6685c-3165-4ae2-8d85-34290ad806df None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:13.750576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:13.751361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:13.751643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:13.751918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:13.758454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:13.759893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-587e77d7-8f13-4a12-8f7a-4ccd720a53f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:13.774496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:13.775721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:13.776807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 582/1165] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 15:00:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:13.782326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:13.792340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:13.880836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10495df8-e3f6-4da7-a93a-cf94d5839e37 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:13.914422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:13.915070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:13.915355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:13.915633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:13.916343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:13.916678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:13.918051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a38f38-e134-44d2-b074-0f98f4885e72 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:13.925641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 584/1166] 10.222.0.41 () {56 vars in 1115 bytes} [Fri May 27 15:00:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.115581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8709b4-3e3b-4c26-9e6c-00d8bcf361d6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.116151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8709b4-3e3b-4c26-9e6c-00d8bcf361d6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.116151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8709b4-3e3b-4c26-9e6c-00d8bcf361d6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.117557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 583/1167] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:16.122935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5b0e-77a1-4036-b4ef-9e6f5414993e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.123330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5b0e-77a1-4036-b4ef-9e6f5414993e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.123518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f5b0e-77a1-4036-b4ef-9e6f5414993e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.140737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a27f5b0e-77a1-4036-b4ef-9e6f5414993e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:16.141971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a27f5b0e-77a1-4036-b4ef-9e6f5414993e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:16.206190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 585/1168] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:16.229621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8709b4-3e3b-4c26-9e6c-00d8bcf361d6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.300393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.331598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8709b4-3e3b-4c26-9e6c-00d8bcf361d6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.365206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.365895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.366127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.366402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.367124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.368494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504d527-7908-453e-b1c5-e47dc79beda9 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.382875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 584/1169] 10.222.0.41 () {60 vars in 1138 bytes} [Fri May 27 15:00:16 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:18.692418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7784-c806-4fbf-ba93-558737e086bf None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.692966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7784-c806-4fbf-ba93-558737e086bf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.692966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d7784-c806-4fbf-ba93-558737e086bf None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.694329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 586/1170] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:18.699650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.700042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.700209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.717919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:18.719083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:18.784319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 585/1171] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:00:18.807704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7784-c806-4fbf-ba93-558737e086bf None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:18.810656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:18.909155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d7784-c806-4fbf-ba93-558737e086bf None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:18.942245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:18.943140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.943372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.943645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] PATH_INFO: `/v3/domains/83409eca73704862a14a4fd9c3ea2b4d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.954624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=83409eca73704862a14a4fd9c3ea2b4d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:18.956071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a600e-d626-454d-8c4f-8d8d351954b9 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:18.957715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 587/1172] 10.222.0.41 () {56 vars in 1183 bytes} [Fri May 27 15:00:18 2022] GET /identity/v3/domains/83409eca73704862a14a4fd9c3ea2b4d => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:18.963592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:19.024490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:19.065378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e013-5402-414c-9f24-8ca40e1ba738 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:19.099265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:19.099919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:19.100226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:19.100555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:19.101291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:19.102779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955cd4-62f6-4ab7-9e22-be68a5e22503 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:19.128800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 586/1173] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 15:00:18 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:21.438112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ab07c-2f87-45b0-bbfa-f1d057ea0a0d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.438648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ab07c-2f87-45b0-bbfa-f1d057ea0a0d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.438648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ab07c-2f87-45b0-bbfa-f1d057ea0a0d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.440043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 588/1174] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:21.445387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.445821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.445960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.463347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:21.464840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:21.532295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 587/1175] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:21.559356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ab07c-2f87-45b0-bbfa-f1d057ea0a0d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.647398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.693397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ab07c-2f87-45b0-bbfa-f1d057ea0a0d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.737218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.738074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.738354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.738626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] PATH_INFO: `/v3/domains/83409eca73704862a14a4fd9c3ea2b4d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.739784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=83409eca73704862a14a4fd9c3ea2b4d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.741242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a2cbe-c3a7-4429-a554-f6eff3f12ebf None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.742861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 589/1176] 10.222.0.41 () {56 vars in 1183 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/domains/83409eca73704862a14a4fd9c3ea2b4d => generated 267 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:21.749616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.795787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.852512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431d444-b03d-4df6-bc75-b577b15454ae None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.887283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.887952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.888911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.889337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.890073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.891588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031d7e5-b241-44c8-a702-e14c23be26c0 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.911096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 588/1177] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:24.210113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5200-2784-42f8-9c83-289c4d42e9bd None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.210677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5200-2784-42f8-9c83-289c4d42e9bd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.210677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb5200-2784-42f8-9c83-289c4d42e9bd None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.212051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 590/1178] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:24.218240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.218634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.218800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.235138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:24.236305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:24.301685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 589/1179] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:24.325256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb5200-2784-42f8-9c83-289c4d42e9bd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:24.426319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb5200-2784-42f8-9c83-289c4d42e9bd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.454678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:24.460895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.461626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.461913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.462195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.472902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:24.480558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:24.482134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:00:24.483039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 591/1180] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:24.488594 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:24.519639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:24.590795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843097-a9cb-4168-aa75-518a3bd09ace None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.625636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.626294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.626606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.626887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.627672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:24.627980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:24.629408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:24.637615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 590/1181] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:24.644051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:24.742699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2924-88b2-4c29-bc0a-f9a425608ca7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.776823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.777682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.777965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.778240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] PATH_INFO: `/v3/users/6ffe2543e0e544cd88a8141828720fb2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.792343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] RBAC: Authorizing `identity:get_user(user_id=6ffe2543e0e544cd88a8141828720fb2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:24.793830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:24.795526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 592/1182] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/users/6ffe2543e0e544cd88a8141828720fb2 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:24.801310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:24.892334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:24.901076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55a5b-1e70-4b3d-bd98-1ab1917abcbc None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.934820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.935629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.935911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.936186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.002264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.003629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.090362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.100351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 591/1183] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:00:25.106144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.155017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.205431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230ffa8-9d7e-4a1f-ac46-80faebb1e206 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.239235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.240087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.240365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.240665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] PATH_INFO: `/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.241884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] RBAC: Authorizing `identity:get_project(project_id=7c78b2dad7e340138f3d6e343aea5d6c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.243322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d9591-8f64-4c88-b33c-fc4319b57287 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.245047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 593/1184] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.251732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.348525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18d12-5653-44c6-baa9-3a88ae676dd5 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.382149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.382810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.383093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.383368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.384491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=7c78b2dad7e340138f3d6e343aea5d6c, user.id=6ffe2543e0e544cd88a8141828720fb2) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:25.384585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.385941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00f0d4-e1e2-4994-bdbb-fc014d23e3ee None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.391660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 592/1185] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/role_assignments?user.id=6ffe2543e0e544cd88a8141828720fb2&scope.project.id=7c78b2dad7e340138f3d6e343aea5d6c&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:27.613086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c78b-5303-4a72-88fa-dea4702e8119 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:27.613635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c78b-5303-4a72-88fa-dea4702e8119 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:27.613635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c78b-5303-4a72-88fa-dea4702e8119 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:27.615008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 594/1186] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:27.620131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:27.620571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:27.620723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:27.638154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:27.639266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:27.689306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:27.706351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 593/1187] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:00:27.729423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546c78b-5303-4a72-88fa-dea4702e8119 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:27.838749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546c78b-5303-4a72-88fa-dea4702e8119 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:27.855290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:27.873495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:27.874283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:27.874571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:27.874848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:27.884925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:27.893325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:27.894441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:00:27.895470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 595/1188] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:27.900517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:27.997021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6533-6d11-4753-830b-2f7c527d1295 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:28.030762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:28.031416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.031700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.031978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.032742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:28.033050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:28.034423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:28.042253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 594/1189] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:28.048723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:28.147101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea349-27f4-4afd-bb4d-b10b3979d96e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:28.180344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:28.181229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.181515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.181808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] PATH_INFO: `/v3/users/6ffe2543e0e544cd88a8141828720fb2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.182991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] RBAC: Authorizing `identity:get_user(user_id=6ffe2543e0e544cd88a8141828720fb2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:28.184487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:28.186812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 596/1190] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:28 2022] GET /identity/v3/users/6ffe2543e0e544cd88a8141828720fb2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:28.192467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:28.241611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:28.292314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc1d027-b82a-4001-82ff-843a027c1d55 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:28.325226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:28.326159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.326446 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.326725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] PATH_INFO: `/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.327904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] RBAC: Authorizing `identity:get_project(project_id=7c78b2dad7e340138f3d6e343aea5d6c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:28.329367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83082876-d2c7-4f93-b535-405b04f3c2fd None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:28.331059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 595/1191] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:28 2022] GET /identity/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:28.336881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:28.385253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:28.452508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b818-fa8f-44c8-9853-756060cb257a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:28.485876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:28.486754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.487036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.487314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] PATH_INFO: `/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c/users/6ffe2543e0e544cd88a8141828720fb2/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.489275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c78b2dad7e340138f3d6e343aea5d6c, user_id=6ffe2543e0e544cd88a8141828720fb2, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:28.490718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17194f94-8135-4b20-821d-f5261828cac8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:28.503017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 597/1192] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 15:00:28 2022] PUT /identity/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c/users/6ffe2543e0e544cd88a8141828720fb2/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:30.663455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe845e-4627-4097-848f-4c41cbb77da9 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.664020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe845e-4627-4097-848f-4c41cbb77da9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.664020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe845e-4627-4097-848f-4c41cbb77da9 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.665369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 596/1193] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:30.671284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.671697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.671848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.688653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:30.689833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:30.755756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 598/1194] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:00:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:00:30.779514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe845e-4627-4097-848f-4c41cbb77da9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.880429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe845e-4627-4097-848f-4c41cbb77da9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.896830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.914458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.915183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.915464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.915737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.926738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:30.934164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:30.935466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:00:30.937236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 597/1195] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:30.941885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.974355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:31.040144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944cc28c-8f4a-47ec-b1fe-8b3f04840a77 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:31.074587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:31.075756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:31.076061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:31.076337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:31.077111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:31.077423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:31.078803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:31.086738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 599/1196] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:31.093504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:31.191325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b372d7-4e89-46ff-9c9e-7dfb32b964fd None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:31.225045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:31.225906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:31.226190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:31.226468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] PATH_INFO: `/v3/users/6ffe2543e0e544cd88a8141828720fb2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:31.227644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] RBAC: Authorizing `identity:get_user(user_id=6ffe2543e0e544cd88a8141828720fb2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:31.229107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:31.230720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 598/1197] 10.222.0.41 () {56 vars in 1177 bytes} [Fri May 27 15:00:31 2022] GET /identity/v3/users/6ffe2543e0e544cd88a8141828720fb2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:31.236497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:31.329774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:31.335821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeb19f-328b-4675-bb2e-774a7bc19bd8 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:31.367892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:31.368557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:31.368844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:31.369121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:31.435853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:31.437335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:31.526726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:31.534073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 600/1198] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:00:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:00:31.539965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:31.584931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:31.639183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ab7c-5c39-488f-924d-21db31a0171f None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:31.672738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:31.673582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:31.673864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:31.674139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] PATH_INFO: `/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:31.675294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] RBAC: Authorizing `identity:get_project(project_id=7c78b2dad7e340138f3d6e343aea5d6c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:31.676758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102822ce-dd2d-445a-90c1-e7ee5774554f None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:31.678412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 599/1199] 10.222.0.41 () {56 vars in 1186 bytes} [Fri May 27 15:00:31 2022] GET /identity/v3/projects/7c78b2dad7e340138f3d6e343aea5d6c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:31.684385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:31.781323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c68e6-db99-4092-87ce-b62026447ed7 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:31.813698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:31.814335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:31.814615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:31.814889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:31.815698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=7c78b2dad7e340138f3d6e343aea5d6c, user.id=6ffe2543e0e544cd88a8141828720fb2) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:31.816007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:31.817459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea063269-c750-41de-9c33-7b0f9cca4a1e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:31.823063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 601/1200] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:00:31 2022] GET /identity/v3/role_assignments?user.id=6ffe2543e0e544cd88a8141828720fb2&scope.project.id=7c78b2dad7e340138f3d6e343aea5d6c&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:41.668117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=44 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:41.820539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=70 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:32.398161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc7f1-165a-4d3a-9e2f-e76850159fa2 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.398161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc7f1-165a-4d3a-9e2f-e76850159fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.398161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fc7f1-165a-4d3a-9e2f-e76850159fa2 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.398161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 600/1201] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:32.405075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.405821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.405821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.424421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:32.425650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:32.511858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 602/1202] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:32.543045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fc7f1-165a-4d3a-9e2f-e76850159fa2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.674047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fc7f1-165a-4d3a-9e2f-e76850159fa2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:32.717477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:32.717477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.717477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.717477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.728665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:32.730465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:32.753534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:32.755271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:32.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:32.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:32.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:32.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:32.756260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 601/1203] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:32.762233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.894899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e3d16-9abf-4952-8b2e-acc43c9f8ee1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:32.932440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:32.939689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:32.940537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.940857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.941141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.942007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:32.942354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:32.943919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5a60d-21a1-4f60-89fb-ae6cb144333b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:32.964804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 603/1204] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:32.968628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.042991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:33.101402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddf89e-b461-4b91-8b83-4bc5384fa456 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:33.164285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:33.165528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.166142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.166717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.168378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:33.170345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676e986-ec84-4807-ab19-02bd200843db None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:33.208968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 602/1205] 10.222.0.41 () {60 vars in 1132 bytes} [Fri May 27 15:01:32 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:35.895862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5433f-8fe7-4e32-805a-3f1f80d2624e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.896419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5433f-8fe7-4e32-805a-3f1f80d2624e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.896419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5433f-8fe7-4e32-805a-3f1f80d2624e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.897847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 604/1206] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:35.903684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.904253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.904253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.921394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:35.922722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:35.998360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 603/1207] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:36.024254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5433f-8fe7-4e32-805a-3f1f80d2624e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.163945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5433f-8fe7-4e32-805a-3f1f80d2624e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.205906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.206798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.207156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.207458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.222106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:36.230834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:36.231904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:36.232793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 605/1208] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:36.239074 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.291157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.363020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0930015-ab4d-4dfb-bed8-adf2b10c4234 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.409200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.409899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.410189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.410500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.411416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:36.411730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.413155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.428720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 604/1209] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:36.429642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.497063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.561409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a27fe-e6dc-4528-af7c-64354b9026a2 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.613441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.614268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.614554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.614833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.628324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.629953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:36.651410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:36.655726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:36.660055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:36.660055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:36.660055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.660055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:36.660055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 606/1210] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:36.660767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.773152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.778038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ee58-cf93-41df-86d3-45b3a70bbae3 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.822363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.823066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.823987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.824364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.825408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:36.825793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.828017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.850166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 605/1211] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:36.851444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.969506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224b83d-2734-4831-90c9-b5795a3110fd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.971795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.014076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.014944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.015217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.015533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.025294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.027840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:37.046239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:37.047634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:37.048894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 607/1212] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:37.057100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.187302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b2c9-668d-4c7a-9d4e-cc81b623a29b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.227134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.227985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.228405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.228788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.230513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.230861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.232613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.257476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 606/1213] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.258976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.366176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba8aa6-0c99-4408-ae7d-51e354d78356 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.405913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.406575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.406861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.407143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.442118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.496824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.498724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.623152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 608/1214] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 15:01:37.629127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.650906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.744199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df57d0-0ce0-4b7b-96d6-fdd6669228de None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.784199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.785050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.785472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.785796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.798985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.800574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:37.820020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:37.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:37.822147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:37.822147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:37.822147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:37.822147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:37.822147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 607/1215] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:37.827716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.915991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.964332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0fb68-88d6-4a89-b482-8407cbf113ee None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.007809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.008659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.009038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.009411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.010376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.011477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.013127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.026580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 609/1216] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:38.032395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.117998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.163845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977175f0-995f-48c4-a0fc-5431ad480b13 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.208543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.209604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.209890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.210169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.219519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.221244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:38.236910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:38.238159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:38.239304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:38.241481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 608/1217] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:38.248557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.400157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68570f67-de93-4185-8941-889f4e8ef7da None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.428120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.441626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.442302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.442590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.442869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.443677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.443978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.445465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbcf92-be47-46ea-a035-af0ccd27d18c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.464797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 610/1218] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:38.466724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.610963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d94c9-14ea-442f-b1eb-346bdb859109 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.647811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.655652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.656600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.656974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.657326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.658345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=8a34be8446d14cf1b9a76e9ffd0d4921) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.658819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.660538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f87c4cb-d8bc-4fd3-b600-e01f26e21b88 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.667557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 609/1219] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/role_assignments?user.id=8a34be8446d14cf1b9a76e9ffd0d4921&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.098275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d66d-2d70-41e5-bce3-c9e04291040a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.098887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d66d-2d70-41e5-bce3-c9e04291040a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.098887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d66d-2d70-41e5-bce3-c9e04291040a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.100254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 611/1220] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:41.109615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.109615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.109615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.131203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:41.132439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:41.228488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 610/1221] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:41.255641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5d66d-2d70-41e5-bce3-c9e04291040a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.371212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5d66d-2d70-41e5-bce3-c9e04291040a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.413567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.414366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.414651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.414930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.426764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:41.435886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.437237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:41.438235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 612/1222] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.443897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.564017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcaa6a-5db6-4674-b950-acb202bc0f7e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.588703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.601218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.601994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.602276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.602549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.603296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.603601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.605734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.614944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 611/1223] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.622114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.667719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.734942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe81c-3011-4b8d-a7de-457aad40bcdc None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.775089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.776064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.776922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.777891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.789805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.791577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:41.811207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.812356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.813298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:41.813298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:41.813298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.813298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:41.813298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 613/1224] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.818471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.929842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec902d2-b499-4cc2-a4b1-005f12307db3 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.974082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.974923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.975430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.975788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.976707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.977127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.979263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.993608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 612/1225] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.999857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.115936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.123916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d866de8-1de5-40e0-8b3d-2724af86f507 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.172797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.174216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.174723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.175285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.184063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.185579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:42.201242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.202495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:42.203803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 614/1226] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.208855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.264017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.321797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52efec-38a6-4af7-b1ad-2a60687b72d8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.365684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.366379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.366650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.366925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.367767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.368105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.369548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.389781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 613/1227] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.395433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.506295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26daaa-4657-4571-9703-f2920210cdab None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.542332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.543129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.543413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.543688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.556364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.558579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:42.580310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.581408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.582372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:42.582372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:42.582372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:42.582372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:42.582372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 615/1228] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.588371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.706803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acccc2c-7644-4b2f-89d6-060696285f1d None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.709476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.750564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.751235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.751497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.751768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.752517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.752826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.754188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.766559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 614/1229] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.772139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.782808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.884743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02279363-7f84-4c5f-892c-ba2e8e71ef2e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.920807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.921606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.921887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.922165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.929033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.930491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:42.946830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.948001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:42.954194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 616/1230] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.954802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.072906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6ce7c-87da-44cb-a251-0ac79f9afd6c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.111957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.112658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.112945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.113219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.114005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:43.114316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.115787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6be8c-8abb-46db-94b0-cfd2ab9bbc5b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.140408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 615/1231] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:43.147651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.236172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.274646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3ee60-3ab0-4002-82a1-8925ba57305d None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.312191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.313615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.314023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.314426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/8a34be8446d14cf1b9a76e9ffd0d4921/roles/8cdd0dfc0a0747d699f4336ddfee7e3a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.331819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=8a34be8446d14cf1b9a76e9ffd0d4921, role_id=8cdd0dfc0a0747d699f4336ddfee7e3a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.333449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3a6d-4878-4f10-851b-565b8dd3314e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.348003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 617/1232] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 15:01:43 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/8a34be8446d14cf1b9a76e9ffd0d4921/roles/8cdd0dfc0a0747d699f4336ddfee7e3a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:45.625954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715009-2d01-4bf9-bca9-7f8fe55e0690 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.626482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715009-2d01-4bf9-bca9-7f8fe55e0690 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.626611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9715009-2d01-4bf9-bca9-7f8fe55e0690 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.627903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 616/1233] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:45.634198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.634662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.634776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.655854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:45.657339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:45.728901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 618/1234] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:45.754786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715009-2d01-4bf9-bca9-7f8fe55e0690 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.793795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.897306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9715009-2d01-4bf9-bca9-7f8fe55e0690 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.936704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.937503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.937785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.938062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.949084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:45.957044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:45.958460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:45.959857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 617/1235] 10.222.0.41 () {56 vars in 1102 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:45.964390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.017338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.084641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bbc9b-8fc6-4016-92c9-c142d45f5109 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.122227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.122887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.123175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.123455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.124204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] RBAC: Adding query filter params (name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.124558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.125939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.139483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 619/1236] 10.222.0.41 () {56 vars in 1103 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.141164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.244217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.262982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6fd3-7880-47fe-9786-813f7d3dd214 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.304245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.305228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.305524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.305825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.322781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.324319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:46.343789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:46.344920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:46.346005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:46.346005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:46.346005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:46.346005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:46.346005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 618/1237] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:46.351765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.462946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27305704-8502-4e8d-8f69-e17df45c9687 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.474010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.507528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.508148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.508478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.508757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.509575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.509918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.511915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.527356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 620/1238] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.533516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.650359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f389-2394-4785-984d-ee6f009d5cf3 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.696687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.697928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.698472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.698973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.703861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.710837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.712468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:46.734183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:46.735990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:46.737019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 619/1239] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:46.742274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.859588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcb999-95e0-4500-9e4f-43cbff1d6c0c None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.900726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.901962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.902248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.902551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.903367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.903683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.905282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.926656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 621/1240] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.932880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.050386 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91857409-e193-484d-a16d-35b2c1074d85 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.092954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.093681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.093992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.094274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.177475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.185492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.187157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.298918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 620/1241] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:01:47.304834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.335800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.421435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6a08c-69b8-4a33-944d-22d433a93ce3 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.466523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.467514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.467928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.468404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.481900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.483878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:47.509428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.511096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.512133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:47.512133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:47.512133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:47.512133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:47.512406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 622/1242] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:47.517748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.637853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.644619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3390a-5b2f-40bf-919f-261a15b39364 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.681849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.683141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.684094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.684384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.685205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:47.685534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.686898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.698849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 621/1243] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:47.704687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.794778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.818912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130aba3b-0d43-4e1b-8c84-8fb5c91b41eb None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.860873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.861694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.861977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.862274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.871978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.873672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:47.888854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.889934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:47.890893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:47.894939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 623/1244] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:47.895942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.008796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8ea05-e931-43fe-bfe8-7d7bb9e5bcff None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.049392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.050323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.050759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.051162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.052252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:48.052627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.054510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acefdbd-0133-48df-9c30-6bc223c2ac2a None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.075808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 622/1245] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:48.077109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.198892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d4930-c20a-4860-bf48-53884b56cfc5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.239000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.239654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.239935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.240206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.241055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] RBAC: Adding query filter params (role.id=8cdd0dfc0a0747d699f4336ddfee7e3a, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=8a34be8446d14cf1b9a76e9ffd0d4921) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:48.241369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.242807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab938e-fefc-402a-a33e-31a2f95bd057 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.249852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 624/1246] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:01:48 2022] GET /identity/v3/role_assignments?user.id=8a34be8446d14cf1b9a76e9ffd0d4921&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=8cdd0dfc0a0747d699f4336ddfee7e3a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:50.616816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43857c7-9c0c-45d5-b8e6-37196caa5bd9 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.617365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43857c7-9c0c-45d5-b8e6-37196caa5bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.617365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43857c7-9c0c-45d5-b8e6-37196caa5bd9 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.618786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 623/1247] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:50.624498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.624933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.625068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.630561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:50.645331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:50.646443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:50.719980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 625/1248] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:50.745966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43857c7-9c0c-45d5-b8e6-37196caa5bd9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:50.765320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:50.855530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43857c7-9c0c-45d5-b8e6-37196caa5bd9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:50.895270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:50.896121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.896353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.896660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.908900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:50.916766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:50.917867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:50.918911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:50.921002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 624/1249] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:50.927019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.034983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fba3e-7985-4ac9-82b1-9b3f1965b1ee None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.068870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.075955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.076696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.076985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.077282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.078022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.078335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.079722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.088360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 626/1250] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.094611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.186288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.211581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83913f7e-26f5-4cf9-a53a-e9e25f453458 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.247713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.248617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.248900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.249174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.261722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.263160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:51.281037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:51.282258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:51.283720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:51.283720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:51.283720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:51.283720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:51.283720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 625/1251] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:51.288043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.407082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a57674-0902-4888-b37c-3c3a1aa6721a None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.443529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.444133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.444419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.444727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.445471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.445784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.447229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.461384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 627/1252] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.468072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.576063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be97507-2b4f-4d0f-a4b2-f7fa7416d9b7 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.604130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.612476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.613301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.613586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.613864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.622487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.623993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:51.639667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:51.640915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:51.641869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 626/1253] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:51.646937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.708943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.757418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e96e2-44b9-4f07-aa5b-6c50a1741188 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.793648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.794342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.794627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.794905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.795752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.796073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.797592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.817318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 628/1254] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.823557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.927107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c665f-23eb-4288-bd3a-1f5f4d98bf11 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.961080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.961659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.961946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.962227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.020066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.033713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.035161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.136339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 627/1255] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:01:52.142767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.247097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b48ce3-d7db-4a6a-b0a2-b4d2f5f0f37e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.259932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.281486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.282257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.282538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.282813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.293353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.294792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:52.311906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:52.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:52.314137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:52.314137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:52.314137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:52.314137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:52.314137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 629/1256] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:52.318861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.404846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.417317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44069c9-63b3-4ff5-aa99-561510136faa None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.450918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76e91-0494-4349-94dd-65036948304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.451567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76e91-0494-4349-94dd-65036948304b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.451847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76e91-0494-4349-94dd-65036948304b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.452121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76e91-0494-4349-94dd-65036948304b None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.452857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76e91-0494-4349-94dd-65036948304b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:52.453162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76e91-0494-4349-94dd-65036948304b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.454516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e76e91-0494-4349-94dd-65036948304b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.466397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 628/1257] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:52.472350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.591309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2980cd0-3eea-4fea-abda-ef65ddaa2c53 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.625182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.625955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.626234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.626505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.632976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.634416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:52.648027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:52.649523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:52.650467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 630/1258] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:52.655693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76e91-0494-4349-94dd-65036948304b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.763540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76e91-0494-4349-94dd-65036948304b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.796337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.797017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.797302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.797577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.798352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:52.798663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.800131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed085674-1816-4b4b-a737-afcea5874d85 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.812818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 629/1259] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:52.819126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.845871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.921073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2796fe-e6b2-485c-9bf9-51f2d829caac None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.954065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.954713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.954998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.955276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.956119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=8a34be8446d14cf1b9a76e9ffd0d4921) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:52.956433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.957890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9858176-4b96-4ab1-b9a1-b651e6a5fb9b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.963183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 631/1260] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/role_assignments?user.id=8a34be8446d14cf1b9a76e9ffd0d4921&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.206307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f1a9-bbad-4566-944a-e5792b8c5f8a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.206889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f1a9-bbad-4566-944a-e5792b8c5f8a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.206889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553f1a9-bbad-4566-944a-e5792b8c5f8a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.208234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 630/1261] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:55.213324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.213772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.214174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.231577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:55.232771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:55.300154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 632/1262] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:55.322391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553f1a9-bbad-4566-944a-e5792b8c5f8a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.333153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.426155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553f1a9-bbad-4566-944a-e5792b8c5f8a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.460761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.461476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.461761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.462039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.472405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:55.480243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:55.481348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:55.482195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 631/1263] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:55.487459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.505924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.586029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf0f5d-305a-4b57-9df4-381ad592e333 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.618358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.619005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.619284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.619557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.621269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:55.621578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.622967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.630739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 633/1264] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.637161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.702640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.736414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d585278-4928-447d-a7a2-1b58acde116c None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.769196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.769974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.770257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.770533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.781571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.783004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:55.799761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:55.800837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:55.801868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:55.801868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:55.801868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:55.801868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:55.801868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 632/1265] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:55.806646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.887205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.906525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d980b2c-e7a1-4910-b828-1765a2866066 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.938591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.939239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.939524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.939802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.940603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:55.940914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.942290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.954216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 634/1266] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.959716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.056993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e6530-ad3f-4bc8-b620-f0c2ed58f635 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.076540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.091159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.091946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.092232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.092539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] PATH_INFO: `/v3/users/placement` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.099776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.101245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:56.115855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:56.116850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:56.117846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 633/1267] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:56.122853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.239933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60541212-710d-4c20-87e9-4681cd6c7494 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.273825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.274480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.274761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.275035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.275844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:56.276659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.278104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.286418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.295278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 635/1268] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:56.300903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.399607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33a62-b9d9-4495-98ac-64f2cdbdcdde None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.433859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.434628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.434905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.435176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.446320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.447780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:56.472367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:56.473544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:56.474484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:56.474484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:56.474484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:56.474484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:56.474484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 634/1269] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:56.479859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.577735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f1117-5f7b-4046-ba01-0160776beae0 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.611505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.612159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.612463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.612755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.613467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:56.613775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.615155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.626700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 636/1270] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:56.632248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.663367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.735896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0706f29-0b84-45d4-a379-a9734193eaf8 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.791206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.792050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.792361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.792666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.800578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.802021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:56.816719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:56.817773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:01:56.818902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 635/1271] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:56.824679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.882683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.923767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f17e5-e8a6-40a6-8dbd-88c17f6e4b6e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.956664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.957343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.957623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.957899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.958664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:56.958972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.960400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e1a0f-4b33-4209-952d-744ec2064dfb None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.973736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 637/1272] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:56.980507 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.069197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.078097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2608dfc-426d-4125-abd3-e0ca553ffc97 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.112879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.113746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.114024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.114296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] PATH_INFO: `/v3/projects/e948f94140fc4057832fe02726946abc/users/8a34be8446d14cf1b9a76e9ffd0d4921/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.116200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] RBAC: Authorizing `identity:create_grant(project_id=e948f94140fc4057832fe02726946abc, user_id=8a34be8446d14cf1b9a76e9ffd0d4921, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.117652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcff18-b2bc-4b28-bc9c-003b07a7b39b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.130062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 636/1273] 10.222.0.41 () {58 vars in 1439 bytes} [Fri May 27 15:01:56 2022] PUT /identity/v3/projects/e948f94140fc4057832fe02726946abc/users/8a34be8446d14cf1b9a76e9ffd0d4921/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:59.312358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5f719-2023-4004-9f53-53f552d10cf3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.312925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5f719-2023-4004-9f53-53f552d10cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.313019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5f719-2023-4004-9f53-53f552d10cf3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.314322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 638/1274] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:59.319665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.320095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.320223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.337409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:59.338719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:59.405075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 637/1275] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:01:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:59.428998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5f719-2023-4004-9f53-53f552d10cf3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.531942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5f719-2023-4004-9f53-53f552d10cf3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.549364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.566321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.567046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.567329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.567608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.578247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:59.585789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:59.586889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:59.587888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 639/1276] 10.222.0.41 () {56 vars in 1096 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:59.593885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.673316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.691618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60f723-5a14-4469-8522-6be1728d5079 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.726000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.726660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.726945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.727223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.727991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:59.728300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.729982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.737955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 638/1277] 10.222.0.41 () {56 vars in 1099 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:59.743528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.842773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8dd1f-0965-45ff-8ff5-cf9bbf70aacd None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.875516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.876278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.876587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.876862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.888016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.889487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:59.906167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:59.907222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:59.908314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:59.908314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:59.908314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:59.908314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:01:59.908314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 640/1278] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:59.912930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.010819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65153-eef2-4e0b-b35b-02c9244fe098 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.043205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.043848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.044132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.044412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.045149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:00.045458 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.046814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.059554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 639/1279] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:00.065089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.096184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.176792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5f637-acab-434e-98b3-9f037863cd8b None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.216432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.217262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.217551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.217829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.225429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.226885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:00.241008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:00.242134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:00.243504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 641/1280] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:00.248292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.252059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.346718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731a15f-52ad-4f06-a86f-e4cb172bb09b None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.379951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.380651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.380938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.381217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.382047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:00.382361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.384222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.402119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 640/1281] 10.222.0.41 () {56 vars in 1143 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:00.407727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.497670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.507549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a4a7-bdc2-414f-ae27-d43e40ca12e5 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.542845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.543495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.543782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.544063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.610765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.612160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.699159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.709399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 642/1282] 10.222.0.41 () {58 vars in 1282 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:00.715691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.776053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.814838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02142ea2-5433-4f53-80f9-f0f20165e029 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.848963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.849749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.850034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.850309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.861103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.862481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:00.880295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:00.881344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:00.882418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:00.882418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:00.882418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:00.882418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:02:00.884600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 641/1283] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:00.887902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.984033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddd83f-88df-4eed-b062-7257e14b474e None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.017198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.017837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.018116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.018390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.019098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:01.019404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.020804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.031907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 643/1284] 10.222.0.41 () {56 vars in 1109 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:01.037486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.137464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68704c-18e0-4519-b032-c0d87c9e0986 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.151639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:01.172160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.172998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.173284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.173563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.180064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.181539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:02:01.195188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application with Lock( May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:01.196214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: ERROR keystone.server.flask.application May 27 15:02:01.197510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 642/1285] 10.222.0.41 () {56 vars in 1111 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:01.202784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.252698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:01.302139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493faf5f-51a1-4931-a075-a8af16619602 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.334497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.335151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.335439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.335717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.336522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:01.336844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.338278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee0914-92be-4052-9c48-4eef9dba5f6b None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.350907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 644/1286] 10.222.0.41 () {56 vars in 1148 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:01.357864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.455900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e3ef-9373-4ae8-a662-44243143fb9e None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.489325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.489972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.490259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.490540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.491356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] RBAC: Adding query filter params (role.id=bde34f992bf3449ca0097f2d900a7537, scope.project.id=e948f94140fc4057832fe02726946abc, user.id=8a34be8446d14cf1b9a76e9ffd0d4921) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:01.491673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.493136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad860e0a-5845-4a58-959e-74ba1b96cc68 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.499102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 643/1287] 10.222.0.41 () {56 vars in 1374 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/role_assignments?user.id=8a34be8446d14cf1b9a76e9ffd0d4921&scope.project.id=e948f94140fc4057832fe02726946abc&role.id=bde34f992bf3449ca0097f2d900a7537 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:03.696625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a170570-e360-461d-863a-c45829337585 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.697154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a170570-e360-461d-863a-c45829337585 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.697154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a170570-e360-461d-863a-c45829337585 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.698824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 645/1288] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:03.704020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.704460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.704602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.722041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:03.723208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:03.753176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.792321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 644/1289] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:02:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:03.814260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a170570-e360-461d-863a-c45829337585 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:03.913129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a170570-e360-461d-863a-c45829337585 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.924085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.947482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.948764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.949044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.949319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.949962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.951331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:03.962813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:03.964970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:03.966083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 646/1290] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:03.970885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:04.065869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910e2fb-45d4-4fb7-8eb7-9170f7572a59 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:04.099404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:04.100056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:04.100339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:04.100644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:04.101343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:04.101651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:04.103135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbc847-c10b-4584-a64d-7c1b7cab0a15 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:04.108071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 645/1291] 10.222.0.41 () {56 vars in 1116 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:04.113704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:04.228210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3071b6-b135-49f3-b52b-5d2c06aa5824 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:04.260120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:04.261399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:04.261682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:04.261961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] PATH_INFO: `/v3/services` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:04.262657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] RBAC: Adding query filter params (type=placement) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:04.262966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:04.264356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646e44e-66af-4995-a7fe-3bf8c5cb5dcc None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:04.268734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 647/1292] 10.222.0.41 () {56 vars in 1116 bytes} [Fri May 27 15:02:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:06.438791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74cf6-a136-45fb-b1cb-713583a51150 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.439326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74cf6-a136-45fb-b1cb-713583a51150 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.439326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f74cf6-a136-45fb-b1cb-713583a51150 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.440750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 646/1293] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:06.445766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ba8c7-8e5d-4b98-8f10-bb816d672861 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.446195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ba8c7-8e5d-4b98-8f10-bb816d672861 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.446323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ba8c7-8e5d-4b98-8f10-bb816d672861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.464433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-da4ba8c7-8e5d-4b98-8f10-bb816d672861 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:06.465950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-da4ba8c7-8e5d-4b98-8f10-bb816d672861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:06.495719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:06.533214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 648/1294] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:02:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:06.556423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f74cf6-a136-45fb-b1cb-713583a51150 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:06.586963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:06.657445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f74cf6-a136-45fb-b1cb-713583a51150 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:06.690798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:06.691455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.691745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.692026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.692779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:06.694139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0b0f-dfc1-4d7b-b20f-66ea1a6e4e67 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:06.708029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 647/1295] 10.222.0.41 () {60 vars in 1141 bytes} [Fri May 27 15:02:06 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:08.927205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e6f09-6bc5-4c0e-8094-abe6facb2367 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.927746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e6f09-6bc5-4c0e-8094-abe6facb2367 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.927746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e6f09-6bc5-4c0e-8094-abe6facb2367 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.929174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 649/1296] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:08.934592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.935010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.935149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.951726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:08.952975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:09.033966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 648/1297] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:02:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:09.056495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e6f09-6bc5-4c0e-8094-abe6facb2367 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.161935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e6f09-6bc5-4c0e-8094-abe6facb2367 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.197184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.198447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.198731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.199006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.199647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.201055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:09.212570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:09.214332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:09.215428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 650/1298] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 15:02:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:09.220717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.239014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.319640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56440-e063-4fe5-8b96-3c52457ccfec None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.355551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.356207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.356517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.356800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.357497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:09.357806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.359170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.364222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 649/1299] 10.222.0.41 () {56 vars in 1116 bytes} [Fri May 27 15:02:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:09.370340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.402971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.470549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2cf3-3c65-4321-b8b6-0b08a7b78710 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.503014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.503664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.503953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.504264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.505080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] RBAC: Adding query filter params (interface=public, service_id=6802686b599446e6b40b0aeab3d81db2, region_id=RegionOne) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:09.505396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.506775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba528-d6d7-4a78-aa75-41a264015b1c None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.512198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 651/1300] 10.222.0.41 () {56 vars in 1251 bytes} [Fri May 27 15:02:09 2022] GET /identity/v3/endpoints?service_id=6802686b599446e6b40b0aeab3d81db2&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:09.517772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.599079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.616592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba95e5f-ffae-4963-985c-d693d88ce751 None admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.649588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.650242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.650526 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.651348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.652027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.653422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421e51d-974d-42de-bf43-11a6391d91cc None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.660167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 650/1301] 10.222.0.41 () {56 vars in 1087 bytes} [Fri May 27 15:02:09 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:02:11.852335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec9b3-cc97-4045-9245-d8fec7df1658 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.852950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec9b3-cc97-4045-9245-d8fec7df1658 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.852950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec9b3-cc97-4045-9245-d8fec7df1658 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.855154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 652/1302] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:11.860283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.860765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.860866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.877359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:11.878536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:11.944161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 651/1303] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:02:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:11.967377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ec9b3-cc97-4045-9245-d8fec7df1658 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.060834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.067577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ec9b3-cc97-4045-9245-d8fec7df1658 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.100794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.101572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.101856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.102131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.102821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.104204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:12.115721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:12.117026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application with Lock( May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: ERROR keystone.server.flask.application May 27 15:02:12.117989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 653/1304] 10.222.0.41 () {56 vars in 1117 bytes} [Fri May 27 15:02:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:12.123590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.198600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.221640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9921-4451-4f64-b66d-f6e15f55b782 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.255345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.255997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.256283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.256593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] PATH_INFO: `/v3/services` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.257297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.257608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.258993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c47b5-9b1c-40f6-9107-32558c5dd4cf None admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.264065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 652/1305] 10.222.0.41 () {56 vars in 1116 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:12.269701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.367123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f4cc8-c379-42ec-af0f-ce021225ff15 None admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.400102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.401412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.401692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.401967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.402698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.404142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5bbe2-4c4c-47d7-b08a-cc5141dd0564 None admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.422496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 654/1306] 10.222.0.41 () {60 vars in 1144 bytes} [Fri May 27 15:02:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:22.261203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=30 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:22.418019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:16.648371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034f62-b91d-4207-a7d6-dad73419ea5d None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:16.648944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034f62-b91d-4207-a7d6-dad73419ea5d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:16.648944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32034f62-b91d-4207-a7d6-dad73419ea5d None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:16.656595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb981df-8daa-4648-b45e-670c7357f062 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:16.657067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb981df-8daa-4648-b45e-670c7357f062 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:16.657160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb981df-8daa-4648-b45e-670c7357f062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:16.660381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 653/1307] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:16.737804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1cb981df-8daa-4648-b45e-670c7357f062 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:16.739179 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1cb981df-8daa-4648-b45e-670c7357f062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:16.861320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 655/1308] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:16.903163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:16.903352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:16.903633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:16.905353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 654/1309] 10.222.0.41 () {54 vars in 964 bytes} [Fri May 27 15:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:16.911876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaa3f1-1d7e-4449-adab-411bf7da624a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:16.912554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaa3f1-1d7e-4449-adab-411bf7da624a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:16.912994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afaa3f1-1d7e-4449-adab-411bf7da624a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:17.007200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9afaa3f1-1d7e-4449-adab-411bf7da624a None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:17.008457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9afaa3f1-1d7e-4449-adab-411bf7da624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:17.031421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:17.124288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 656/1310] 10.222.0.41 () {58 vars in 1061 bytes} [Fri May 27 15:03:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:17.135528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:17.137820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:17.320815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f34a0e-72cb-446a-9e3e-0f3c480e83f4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:17.416918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:17.417499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:17.417783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:17.418063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:17.419773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:17.480694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:17.538702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:17.540898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fed89-7fc4-4f44-b72f-c75f89481822 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:17.560970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 655/1311] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:20.107604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:20.108146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:20.108146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:20.109583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 657/1312] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:20.114743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3baa41-0f6f-432d-aef4-0e4a3e14de25 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:20.115151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3baa41-0f6f-432d-aef4-0e4a3e14de25 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:20.115310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3baa41-0f6f-432d-aef4-0e4a3e14de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:20.197484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1e3baa41-0f6f-432d-aef4-0e4a3e14de25 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:20.198603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1e3baa41-0f6f-432d-aef4-0e4a3e14de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:20.297219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 656/1313] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:20.335930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:20.338564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:20.448295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0c2a0-6468-4d05-9403-844b8bb46d48 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:20.521260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:20.525084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:20.525899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:20.526249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:20.526754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:20.528434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:20.640917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:20.642294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097649a4-847f-4bd5-8c86-996fb335ccfd service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:20.661564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 658/1314] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:23.239101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:23.239620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:23.239731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:23.241224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 657/1315] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:23.247269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3d77-94b1-4678-aee8-80f580a2543b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:23.247722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3d77-94b1-4678-aee8-80f580a2543b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:23.247825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3d77-94b1-4678-aee8-80f580a2543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:23.313921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3bdd3d77-94b1-4678-aee8-80f580a2543b None None] MFA Rules not processed for user `f3c65b39372942929f6e010bdbdf024b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:23.315150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3bdd3d77-94b1-4678-aee8-80f580a2543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:23.369038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:23.419149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 659/1316] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:23.457971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:23.459525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:23.470559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:23.572726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7e5bd-2f27-4463-be44-c86f83df66eb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:23.648629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:23.649203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:23.649489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:23.649758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:23.651500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:23.715139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:23.764427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:23.765891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72105cd-ecd8-4454-8c9d-6b659792377e service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:23.786466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 658/1317] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:26.694037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:26.694546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:26.694546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:26.695879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 660/1318] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:26.701644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a2a9-4d23-4cd2-874c-ce1b52693f03 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:26.702059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a2a9-4d23-4cd2-874c-ce1b52693f03 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:26.702203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813a2a9-4d23-4cd2-874c-ce1b52693f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:26.767720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1813a2a9-4d23-4cd2-874c-ce1b52693f03 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:26.768987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1813a2a9-4d23-4cd2-874c-ce1b52693f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:26.867653 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 659/1319] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:26.908528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:26.910098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:27.014417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd717f3-6c71-4014-a876-83c8a5a0661d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:27.087406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:27.088248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:27.088555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:27.088829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:27.090458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:27.098016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:27.204234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:27.206396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999165a-e449-4155-beb0-6d52df319da5 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:27.224214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 661/1320] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:30.114713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:30.115304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:30.115304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:30.116694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 660/1321] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:30.122534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073d07-f685-4ddc-86af-0c2783911cc3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:30.122974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073d07-f685-4ddc-86af-0c2783911cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:30.123439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04073d07-f685-4ddc-86af-0c2783911cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:30.191627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-04073d07-f685-4ddc-86af-0c2783911cc3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:30.192846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-04073d07-f685-4ddc-86af-0c2783911cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:30.252856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:30.292785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 662/1322] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:30.328879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:30.330556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:30.349780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:30.438018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c44115-b3cb-43ee-83e4-96a3e2c84a69 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:30.514926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:30.515582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:30.515864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:30.516139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:30.517808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:30.589488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:30.629760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:30.631211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f032d-76f1-4488-a903-ef163507cda0 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:30.650820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 661/1323] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:32.938238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:32.938778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:32.938778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:32.940239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 663/1324] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:32.945668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2cd2a-8a45-497a-af24-1a114ef7e3e5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:32.946089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2cd2a-8a45-497a-af24-1a114ef7e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:32.946306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2cd2a-8a45-497a-af24-1a114ef7e3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:33.015743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-23c2cd2a-8a45-497a-af24-1a114ef7e3e5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:33.017038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-23c2cd2a-8a45-497a-af24-1a114ef7e3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:33.115854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 662/1325] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:33.156961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:33.158505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:33.264920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199ef74-ebea-4dc6-9d34-bca8a70793d0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:33.337827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:33.338476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:33.338759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:33.339035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:33.340962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:33.358172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:33.455278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:33.456772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e77fc8-f246-4c3f-9205-73e896b139ce service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:33.476093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 664/1326] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:35.749232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:35.749767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:35.749767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:35.751246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 663/1327] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:35.756325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9362-acf3-4fea-a173-d54b6681a57e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:35.757142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9362-acf3-4fea-a173-d54b6681a57e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:35.757427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9362-acf3-4fea-a173-d54b6681a57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:35.826576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1aaa9362-acf3-4fea-a173-d54b6681a57e None None] MFA Rules not processed for user `f3c65b39372942929f6e010bdbdf024b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:35.827800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1aaa9362-acf3-4fea-a173-d54b6681a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:35.895931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:35.928471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 665/1328] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:03:35.967010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:35.968593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:35.999727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:36.077268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8363f9-df22-4a68-9bec-8da16a575eb2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:36.152617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:36.153265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:36.153552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:36.153830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:36.155493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:36.233636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:36.266556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:36.267917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def840f-76ca-4904-9a23-247d6667cb46 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:36.286314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 664/1329] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:38.743362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:38.743933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:38.744274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:38.745756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 666/1330] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:38.751434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6424b8-ef31-4b4b-96dc-b9713635f91d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:38.751962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6424b8-ef31-4b4b-96dc-b9713635f91d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:38.752059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6424b8-ef31-4b4b-96dc-b9713635f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:38.822340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ba6424b8-ef31-4b4b-96dc-b9713635f91d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:38.823542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ba6424b8-ef31-4b4b-96dc-b9713635f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:38.921688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 665/1331] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:38.956420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:38.958011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:39.066171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f022ac8-a02e-4893-899b-efa2f67ad2df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:39.142444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:39.143097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:39.143381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:39.143656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:39.145348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:39.173334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:39.255284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:39.256771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1e9b3-d1b1-4e1e-9f4d-c54a2c8d4d1f service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:39.275780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 667/1332] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:41.560732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277fa-1cac-4a44-8d17-558303dbe0d7 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:41.561177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277fa-1cac-4a44-8d17-558303dbe0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:41.561284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277fa-1cac-4a44-8d17-558303dbe0d7 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:41.562721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 666/1333] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:41.567818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a04630-d229-4186-a570-46f5746367f4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:41.568257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a04630-d229-4186-a570-46f5746367f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:41.568382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a04630-d229-4186-a570-46f5746367f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:41.635137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-58a04630-d229-4186-a570-46f5746367f4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:41.636291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-58a04630-d229-4186-a570-46f5746367f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:41.710234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:41.733370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 668/1334] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:41.789447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:41.790383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:41.790668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:41.792128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 667/1335] 10.222.0.41 () {54 vars in 964 bytes} [Fri May 27 15:03:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:41.798147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a26f-de23-44f6-b385-ab0592e47e1f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:41.798438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a26f-de23-44f6-b385-ab0592e47e1f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:41.798718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6a26f-de23-44f6-b385-ab0592e47e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:41.875951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-00c6a26f-de23-44f6-b385-ab0592e47e1f None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:41.877172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-00c6a26f-de23-44f6-b385-ab0592e47e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:41.951870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 669/1336] 10.222.0.41 () {58 vars in 1061 bytes} [Fri May 27 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:41.960786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:41.962848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:42.002553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:42.067773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9295130-bfc6-4480-a0f8-4253fb8fce6e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:42.142272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:42.142967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:42.143277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:42.143576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:42.145344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:42.232949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:42.259401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:42.260886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa048a-0f53-4067-89ba-00049b2f38ba service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:42.278669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 668/1337] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:45.366749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:45.367348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:45.367348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:45.369396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 670/1338] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:45.374606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af5e6-edf3-4d06-ab90-11bffee7d4dc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:45.375035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af5e6-edf3-4d06-ab90-11bffee7d4dc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:45.375170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50af5e6-edf3-4d06-ab90-11bffee7d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:45.441048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c50af5e6-edf3-4d06-ab90-11bffee7d4dc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:45.442244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c50af5e6-edf3-4d06-ab90-11bffee7d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:45.540191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 669/1339] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:45.577115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:45.578631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:45.656193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:45.685335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d360e6-ff46-4543-bf80-ca2aa2264b89 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:45.760728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:45.761367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:45.761646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:45.761917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:45.763546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:45.875284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:45.876782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d58ee-d227-43e2-b977-1961e00ac82c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:45.894476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 671/1340] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:50.267845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:50.268366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:50.268366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:50.269746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 670/1341] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:50.275821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529372-d17f-4ee9-aa34-9d958f62928a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:50.276268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529372-d17f-4ee9-aa34-9d958f62928a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:50.276370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38529372-d17f-4ee9-aa34-9d958f62928a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:50.282392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:50.350054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-38529372-d17f-4ee9-aa34-9d958f62928a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:50.351236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-38529372-d17f-4ee9-aa34-9d958f62928a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:50.467534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 672/1342] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:50.506156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:50.508870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:50.559905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:50.607713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d82a7f-7bc7-43bd-b7a6-b8700b00ec50 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:50.611511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:50.612153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:50.612434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:50.612739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:50.621951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:50.623370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d85358-fd70-4a2c-9364-2ec0f781b54f service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:50.633012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 671/1343] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:53.452799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969debb-792c-42ef-83b4-9137ba50266b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:53.453318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969debb-792c-42ef-83b4-9137ba50266b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:53.453318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969debb-792c-42ef-83b4-9137ba50266b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:53.454651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 673/1344] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:53.459883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b47af-45e6-4e0c-9ad7-cff150eef506 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:53.460317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b47af-45e6-4e0c-9ad7-cff150eef506 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:53.460467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b47af-45e6-4e0c-9ad7-cff150eef506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:53.488974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-320b47af-45e6-4e0c-9ad7-cff150eef506 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:53.490174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-320b47af-45e6-4e0c-9ad7-cff150eef506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:53.567998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 672/1345] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:53.605895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969debb-792c-42ef-83b4-9137ba50266b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:53.616728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969debb-792c-42ef-83b4-9137ba50266b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:53.620883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:53.621579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:53.621928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:53.622361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:53.631373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:53.632921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13a1aa-1abe-4a07-a06b-fe2889d54595 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:53.643289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 674/1346] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:03:57.816609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229c18-0dac-4ef7-b74c-cfcfdf072b4a None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:57.817487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229c18-0dac-4ef7-b74c-cfcfdf072b4a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:57.817804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f229c18-0dac-4ef7-b74c-cfcfdf072b4a None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:57.819206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 673/1347] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:57.825591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d34a-2ddf-47d1-ab34-1baf29a9418b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:57.826038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d34a-2ddf-47d1-ab34-1baf29a9418b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:57.826189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d34a-2ddf-47d1-ab34-1baf29a9418b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:57.844797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a5b6d34a-2ddf-47d1-ab34-1baf29a9418b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:57.846120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a5b6d34a-2ddf-47d1-ab34-1baf29a9418b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:57.854337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 675/1348] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:03:57.892662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229c18-0dac-4ef7-b74c-cfcfdf072b4a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:57.904156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f229c18-0dac-4ef7-b74c-cfcfdf072b4a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:57.907885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:57.908547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:57.908827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:57.909101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:57.918631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:57.920042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f0ce2-ab9d-4fb6-b010-5feb1caa01bb service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:57.929392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 674/1349] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:03.428726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a5bb-1489-46ca-8718-54f273d6a6e3 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:03.429709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a5bb-1489-46ca-8718-54f273d6a6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:03.430161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a5bb-1489-46ca-8718-54f273d6a6e3 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:03.431976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 676/1350] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:03.437129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581c0b-feae-48c4-b4a8-751fb92b3db3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:03.437924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581c0b-feae-48c4-b4a8-751fb92b3db3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:03.438378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83581c0b-feae-48c4-b4a8-751fb92b3db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:03.513097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-83581c0b-feae-48c4-b4a8-751fb92b3db3 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:03.514513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-83581c0b-feae-48c4-b4a8-751fb92b3db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:03.543808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:03.607860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 675/1351] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:07.839123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=72 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:07.853796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008fac6-0edb-4b6f-af6c-232edaf0fc30 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:07.854684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008fac6-0edb-4b6f-af6c-232edaf0fc30 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:07.855414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008fac6-0edb-4b6f-af6c-232edaf0fc30 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:07.857256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 677/1352] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:04:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:07.862599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a170-3201-4f0e-8692-b3873211a233 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:07.863447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a170-3201-4f0e-8692-b3873211a233 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:07.863806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a170-3201-4f0e-8692-b3873211a233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:07.883504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-73e8a170-3201-4f0e-8692-b3873211a233 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:07.884873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-73e8a170-3201-4f0e-8692-b3873211a233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:07.893539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 676/1353] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:09.546923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cccff-8771-4ad9-ac00-f6871ba9971e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:09.547444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cccff-8771-4ad9-ac00-f6871ba9971e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:09.547558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8cccff-8771-4ad9-ac00-f6871ba9971e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:09.548908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 678/1354] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:09.553855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357480-c136-48f1-93ea-7a34c44a69a5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:09.554300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357480-c136-48f1-93ea-7a34c44a69a5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:09.554485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357480-c136-48f1-93ea-7a34c44a69a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:09.573558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f1357480-c136-48f1-93ea-7a34c44a69a5 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:09.574759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f1357480-c136-48f1-93ea-7a34c44a69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:09.583358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 677/1355] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:04:13.643429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bd95d-0b44-4b91-9135-80de1df75c7e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.644049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bd95d-0b44-4b91-9135-80de1df75c7e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.644049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bd95d-0b44-4b91-9135-80de1df75c7e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.645385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 679/1356] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:13.651298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1b34-4192-4390-9c41-9eca067c2665 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.651694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1b34-4192-4390-9c41-9eca067c2665 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.651890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d1b34-4192-4390-9c41-9eca067c2665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.676768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-127d1b34-4192-4390-9c41-9eca067c2665 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:13.677948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-127d1b34-4192-4390-9c41-9eca067c2665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:13.686749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 678/1357] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:13.740653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70912c7-59a5-40be-bb3a-61a3b11ea8d8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.741132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70912c7-59a5-40be-bb3a-61a3b11ea8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.741580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70912c7-59a5-40be-bb3a-61a3b11ea8d8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.744135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 680/1358] 10.222.0.41 () {54 vars in 961 bytes} [Fri May 27 15:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:13.752409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4059aa-ac47-4d8e-ba9d-aa01b6c7a1a7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.753093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4059aa-ac47-4d8e-ba9d-aa01b6c7a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.753513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4059aa-ac47-4d8e-ba9d-aa01b6c7a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.810515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fd4059aa-ac47-4d8e-ba9d-aa01b6c7a1a7 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:13.811797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fd4059aa-ac47-4d8e-ba9d-aa01b6c7a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:13.906697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 679/1359] 10.222.0.41 () {58 vars in 1058 bytes} [Fri May 27 15:04:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:13.915201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70912c7-59a5-40be-bb3a-61a3b11ea8d8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:13.931250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70912c7-59a5-40be-bb3a-61a3b11ea8d8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:13.938225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:13.938874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.939161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.939440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.949725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:13.951589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704bf3b-7714-4f39-9cc7-5b20771c0c93 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:13.962287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 681/1360] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:04:16.420744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532c90c-f406-4ed1-97d9-866cfd2576db None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.421261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532c90c-f406-4ed1-97d9-866cfd2576db None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.421379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532c90c-f406-4ed1-97d9-866cfd2576db None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.422808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 680/1361] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:04:16.428091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a404d-b34c-4d4d-8039-067712317635 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.428508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a404d-b34c-4d4d-8039-067712317635 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.428607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a404d-b34c-4d4d-8039-067712317635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.456426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-234a404d-b34c-4d4d-8039-067712317635 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:16.457947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-234a404d-b34c-4d4d-8039-067712317635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:16.467402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 682/1362] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:16.514670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532c90c-f406-4ed1-97d9-866cfd2576db None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:16.530342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532c90c-f406-4ed1-97d9-866cfd2576db None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:16.536010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:16.536776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.537127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.537470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.547167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:16.549015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e40e2-e884-48a0-97ac-53a7d02e2136 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:16.560542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 681/1363] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:19.121743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6a81-ee42-4943-83a8-a026c6bc011b None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:19.122265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6a81-ee42-4943-83a8-a026c6bc011b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:19.122354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f6a81-ee42-4943-83a8-a026c6bc011b None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:19.123666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 683/1364] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:19.129227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd693-6bc4-4345-ae7b-b39d2458c8c1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:19.129695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd693-6bc4-4345-ae7b-b39d2458c8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:19.129803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd693-6bc4-4345-ae7b-b39d2458c8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:19.162266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-43ddd693-6bc4-4345-ae7b-b39d2458c8c1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:19.163462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43ddd693-6bc4-4345-ae7b-b39d2458c8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:19.173375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 682/1365] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:19.221400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2afc-f007-40ef-b132-714523d1a6a1 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:19.221620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2afc-f007-40ef-b132-714523d1a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:19.221901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe2afc-f007-40ef-b132-714523d1a6a1 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:19.223309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 684/1366] 10.222.0.41 () {54 vars in 961 bytes} [Fri May 27 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:19.229339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecac86-4281-496f-97ef-b84b419de3a9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:19.229803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecac86-4281-496f-97ef-b84b419de3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:19.230092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecac86-4281-496f-97ef-b84b419de3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:19.248915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f3ecac86-4281-496f-97ef-b84b419de3a9 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:19.250094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f3ecac86-4281-496f-97ef-b84b419de3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:19.259828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 683/1367] 10.222.0.41 () {58 vars in 1058 bytes} [Fri May 27 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:19.267809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe2afc-f007-40ef-b132-714523d1a6a1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:19.283291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe2afc-f007-40ef-b132-714523d1a6a1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:19.288837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:19.290115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:19.290472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:19.290821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:19.299244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:19.300883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092b738-230e-4681-9e3f-d8031b9cb445 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:19.310431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 685/1368] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:21.635236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0e14-bf28-408e-ba20-8fe6c1184efb None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.636227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0e14-bf28-408e-ba20-8fe6c1184efb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.636840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0e14-bf28-408e-ba20-8fe6c1184efb None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.642238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 684/1369] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:21.643921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5bc0-5b1b-45b7-b3da-50fe58856950 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.644325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5bc0-5b1b-45b7-b3da-50fe58856950 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.644497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a5bc0-5b1b-45b7-b3da-50fe58856950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.668248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ec6a5bc0-5b1b-45b7-b3da-50fe58856950 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:21.670123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ec6a5bc0-5b1b-45b7-b3da-50fe58856950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:21.679001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 686/1370] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:21.731748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d0e14-bf28-408e-ba20-8fe6c1184efb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:21.747897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d0e14-bf28-408e-ba20-8fe6c1184efb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:21.756395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:21.757162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.757428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.757969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.767921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:21.769625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7a819-bcf8-494f-983b-49c4714a8546 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:21.793494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 685/1371] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:21.967149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31a9c-b09f-4282-a841-6f7131a11d85 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.967351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31a9c-b09f-4282-a841-6f7131a11d85 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.967650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd31a9c-b09f-4282-a841-6f7131a11d85 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.969114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 687/1372] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:21.975552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e2a-4c6b-4087-ad2d-2d4c2c4a65cf None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.975879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e2a-4c6b-4087-ad2d-2d4c2c4a65cf None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.976130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f061e2a-4c6b-4087-ad2d-2d4c2c4a65cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.997891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4f061e2a-4c6b-4087-ad2d-2d4c2c4a65cf None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:21.999416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4f061e2a-4c6b-4087-ad2d-2d4c2c4a65cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:22.008646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 686/1373] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:22.289400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab8268-b208-49f1-ad1c-4e4bb8b4a8a6 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.289942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab8268-b208-49f1-ad1c-4e4bb8b4a8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.290388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab8268-b208-49f1-ad1c-4e4bb8b4a8a6 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.291936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 688/1374] 10.222.0.41 () {54 vars in 963 bytes} [Fri May 27 15:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:22.297415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6872b-0dcd-4074-b590-2620a9b5fea1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.297798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6872b-0dcd-4074-b590-2620a9b5fea1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.298153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6872b-0dcd-4074-b590-2620a9b5fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.305882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:22.351023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a4a6872b-0dcd-4074-b590-2620a9b5fea1 None None] MFA Rules not processed for user `c1a532f2e8b446b7a03b38b3dda377a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:22.352815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a4a6872b-0dcd-4074-b590-2620a9b5fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:22.422127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 687/1375] 10.222.0.41 () {58 vars in 1060 bytes} [Fri May 27 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:22.430154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab8268-b208-49f1-ad1c-4e4bb8b4a8a6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.441572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab8268-b208-49f1-ad1c-4e4bb8b4a8a6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.446009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.446591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.446943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.447292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.457765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.459449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759cd45-176e-4199-aea9-4509a32109cd service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.470698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 689/1376] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:24.112437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183258c7-f374-41f6-8898-75193c435a74 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:24.113055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183258c7-f374-41f6-8898-75193c435a74 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:24.113055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183258c7-f374-41f6-8898-75193c435a74 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:24.114438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 688/1377] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:24.121015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76535c-40bf-41bf-89c1-0c3bd19aec53 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:24.121563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76535c-40bf-41bf-89c1-0c3bd19aec53 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:24.121669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76535c-40bf-41bf-89c1-0c3bd19aec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:24.149393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5c76535c-40bf-41bf-89c1-0c3bd19aec53 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:24.150623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c76535c-40bf-41bf-89c1-0c3bd19aec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:24.159031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 690/1378] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:24.214422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183258c7-f374-41f6-8898-75193c435a74 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:24.229909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183258c7-f374-41f6-8898-75193c435a74 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:24.235372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:24.236043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:24.236325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:24.236635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:24.246462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:24.248103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7748c-3081-4fd4-bac2-c5ce7fa81bbc service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:24.259450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 689/1379] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:26.536666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2efc-3926-44b5-814f-ba67f1929f44 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.537214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2efc-3926-44b5-814f-ba67f1929f44 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.537214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2efc-3926-44b5-814f-ba67f1929f44 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.538601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 691/1380] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:26.544308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d387-6ac4-484e-b700-bd489a7bec2d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.544774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d387-6ac4-484e-b700-bd489a7bec2d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.545106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d387-6ac4-484e-b700-bd489a7bec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.563350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7937d387-6ac4-484e-b700-bd489a7bec2d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:26.564722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7937d387-6ac4-484e-b700-bd489a7bec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:26.573729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 690/1381] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:26.625402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a2efc-3926-44b5-814f-ba67f1929f44 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.641851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a2efc-3926-44b5-814f-ba67f1929f44 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.647861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.649417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.649894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.650279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.659815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:26.661563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef475d-d740-466e-830f-bb8d64394150 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:26.672338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 692/1382] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:29.003353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a03-f343-45a6-90f8-00826c6d8873 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:29.003872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a03-f343-45a6-90f8-00826c6d8873 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:29.003872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a03-f343-45a6-90f8-00826c6d8873 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:29.005226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 691/1383] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:04:29.011352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3499bf-5bf3-4440-885a-f4fc9b0be8e0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:29.012206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3499bf-5bf3-4440-885a-f4fc9b0be8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:29.012666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3499bf-5bf3-4440-885a-f4fc9b0be8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:29.034259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7e3499bf-5bf3-4440-885a-f4fc9b0be8e0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:29.035572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7e3499bf-5bf3-4440-885a-f4fc9b0be8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:29.045053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 693/1384] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:29.089822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93a03-f343-45a6-90f8-00826c6d8873 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:29.105866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93a03-f343-45a6-90f8-00826c6d8873 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:29.111376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:29.112043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:29.112332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:29.112648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:29.122268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:29.123901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5444-37ec-4a0c-907e-2db6ae9ec4e6 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:29.141699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 692/1385] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:31.403264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea9f0-4f8a-4ba6-ac78-6895c2cd8e26 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.403908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea9f0-4f8a-4ba6-ac78-6895c2cd8e26 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.403908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ea9f0-4f8a-4ba6-ac78-6895c2cd8e26 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.405251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 694/1386] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:31.410633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88fcb-e85f-45d4-a3f1-804dc5703f6f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.411173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88fcb-e85f-45d4-a3f1-804dc5703f6f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.411173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb88fcb-e85f-45d4-a3f1-804dc5703f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.429732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ccb88fcb-e85f-45d4-a3f1-804dc5703f6f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:31.430965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ccb88fcb-e85f-45d4-a3f1-804dc5703f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:31.439336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 693/1387] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:31.485840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ea9f0-4f8a-4ba6-ac78-6895c2cd8e26 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.501088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ea9f0-4f8a-4ba6-ac78-6895c2cd8e26 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.506497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.507148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.507434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.508294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.516758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:31.518373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b2887-a68d-46ba-8eee-60d7fd52efd1 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:31.528454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 695/1388] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:33.910146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed571-d741-47e5-b8d3-ae3d278ad90f None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:33.910709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed571-d741-47e5-b8d3-ae3d278ad90f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:33.910709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ed571-d741-47e5-b8d3-ae3d278ad90f None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:33.912151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 694/1389] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:33.920191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c772-837a-431d-9457-dc61cbe91215 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:33.920618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c772-837a-431d-9457-dc61cbe91215 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:33.920944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5c772-837a-431d-9457-dc61cbe91215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:33.943972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-03b5c772-837a-431d-9457-dc61cbe91215 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:33.946312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-03b5c772-837a-431d-9457-dc61cbe91215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:33.956745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 696/1390] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:34.014859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ed571-d741-47e5-b8d3-ae3d278ad90f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:34.030905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ed571-d741-47e5-b8d3-ae3d278ad90f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:34.036654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:34.037312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:34.037603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:34.038063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:34.047652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:34.049329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d58a4-0af5-425e-9377-157dea0f92bf service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:34.060874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 695/1391] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:36.407924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc6ab4-eea6-4af7-b8f1-b29397bdc845 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.408505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc6ab4-eea6-4af7-b8f1-b29397bdc845 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.408505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc6ab4-eea6-4af7-b8f1-b29397bdc845 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.409921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 697/1392] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:36.416019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f83d7-a9fc-40bc-80be-fa707237ede0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.416511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f83d7-a9fc-40bc-80be-fa707237ede0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.416674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f83d7-a9fc-40bc-80be-fa707237ede0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.435708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-510f83d7-a9fc-40bc-80be-fa707237ede0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:36.436981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-510f83d7-a9fc-40bc-80be-fa707237ede0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:36.445670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 696/1393] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:36.490429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc6ab4-eea6-4af7-b8f1-b29397bdc845 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.505798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc6ab4-eea6-4af7-b8f1-b29397bdc845 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.511851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.512551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.512846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.513129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.521606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.523759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132a4f-872b-419d-8a14-bac4f36919b8 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.533736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 698/1394] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:38.531194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db90-08cb-4fe3-9f13-82f00d9f16e8 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.531716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db90-08cb-4fe3-9f13-82f00d9f16e8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.531851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017db90-08cb-4fe3-9f13-82f00d9f16e8 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.533174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 697/1395] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:38.539070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10cf4-3367-4568-9d2e-5616e7694e9e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.539523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10cf4-3367-4568-9d2e-5616e7694e9e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.539641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd10cf4-3367-4568-9d2e-5616e7694e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.546080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=52 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:38.559873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fbd10cf4-3367-4568-9d2e-5616e7694e9e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:38.561112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fbd10cf4-3367-4568-9d2e-5616e7694e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:38.572492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 699/1396] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:38.632561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b8e5-760c-4580-9549-c472fd6d8493 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.632867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b8e5-760c-4580-9549-c472fd6d8493 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.633117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b8e5-760c-4580-9549-c472fd6d8493 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.636225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 698/1397] 10.222.0.41 () {54 vars in 962 bytes} [Fri May 27 15:04:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:38.642990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb433-4b5e-46bb-b8f3-80ff4c1db116 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.643356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb433-4b5e-46bb-b8f3-80ff4c1db116 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.643744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fb433-4b5e-46bb-b8f3-80ff4c1db116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.692918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3e2fb433-4b5e-46bb-b8f3-80ff4c1db116 None None] MFA Rules not processed for user `36672cdeb2f148698b116e06095955d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:38.694101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e2fb433-4b5e-46bb-b8f3-80ff4c1db116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:38.760751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 700/1398] 10.222.0.41 () {58 vars in 1059 bytes} [Fri May 27 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:38.768847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b8e5-760c-4580-9549-c472fd6d8493 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:38.780213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b8e5-760c-4580-9549-c472fd6d8493 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:38.783669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:38.784300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.784625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.784981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.786741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b949c-a721-4152-af9b-d87bc46c1b3e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.787039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b949c-a721-4152-af9b-d87bc46c1b3e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.787316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b949c-a721-4152-af9b-d87bc46c1b3e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.789515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 701/1399] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:04:38.794132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155f45-02a2-4351-8032-9f41d66c9aac None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.794417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155f45-02a2-4351-8032-9f41d66c9aac None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.794725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155f45-02a2-4351-8032-9f41d66c9aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.795857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:38.797324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:38.807002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 699/1400] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:38.814046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2b155f45-02a2-4351-8032-9f41d66c9aac None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:38.815228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2b155f45-02a2-4351-8032-9f41d66c9aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:38.823506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 702/1401] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:04:38.870515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:38.882743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb463-b003-4db8-8f24-1da5b6c7f3ce service glance] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:38.888416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:38.889026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:38.889311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:38.889587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:38.897961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:38.900091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f981c-ee18-4252-bbb2-f900438ab00c service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:38.909494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 700/1402] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:39.350490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00ac5-c118-4968-8a23-0f600a61da6c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.350787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00ac5-c118-4968-8a23-0f600a61da6c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.351379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa00ac5-c118-4968-8a23-0f600a61da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.405924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6fa00ac5-c118-4968-8a23-0f600a61da6c None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:39.407047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6fa00ac5-c118-4968-8a23-0f600a61da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:39.474927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 703/1403] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:39.520366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95472f0f-c7c7-468d-9009-5cb756685b6c None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.520693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95472f0f-c7c7-468d-9009-5cb756685b6c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.520975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95472f0f-c7c7-468d-9009-5cb756685b6c None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.522356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 701/1404] 10.222.0.41 () {54 vars in 942 bytes} [Fri May 27 15:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:39.529741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de88c-4897-4690-a41c-41876118b6e5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.530035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de88c-4897-4690-a41c-41876118b6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.530313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de88c-4897-4690-a41c-41876118b6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.580650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-504de88c-4897-4690-a41c-41876118b6e5 None None] MFA Rules not processed for user `5b7e595d067a42dca56f0ef5fe8485c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:39.581833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-504de88c-4897-4690-a41c-41876118b6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:39.590416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:39.657185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 704/1405] 10.222.0.41 () {58 vars in 1039 bytes} [Fri May 27 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:39.666764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95472f0f-c7c7-468d-9009-5cb756685b6c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:39.680818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95472f0f-c7c7-468d-9009-5cb756685b6c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:39.685785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:39.686432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.686716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.686995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.695977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:39.697671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689fe2e6-2513-48b0-b2b0-ab3b530971dd service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:39.705895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 702/1406] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:04:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:04:41.199608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a3334-2795-4582-8e7f-0949278e1f34 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.200134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a3334-2795-4582-8e7f-0949278e1f34 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.200134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a3334-2795-4582-8e7f-0949278e1f34 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.201522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 705/1407] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:41.207039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47dd26-2d61-45b6-a6b7-b5d10ee409eb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.207542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47dd26-2d61-45b6-a6b7-b5d10ee409eb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.207647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47dd26-2d61-45b6-a6b7-b5d10ee409eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.226980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-5f47dd26-2d61-45b6-a6b7-b5d10ee409eb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:41.228359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5f47dd26-2d61-45b6-a6b7-b5d10ee409eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:41.237225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 703/1408] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:04:41.284044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a3334-2795-4582-8e7f-0949278e1f34 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:41.299049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a3334-2795-4582-8e7f-0949278e1f34 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.304543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.305340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.305627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.305908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.314248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.315868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aace6c-235b-49ed-bc71-079ae9096d16 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.326910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 706/1409] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:04:51.220805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:51.322124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=33 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:03.182349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147681-7772-4870-b9b3-e61657b1972e None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:03.183006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147681-7772-4870-b9b3-e61657b1972e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:03.183006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38147681-7772-4870-b9b3-e61657b1972e None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:03.184301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 704/1410] 10.222.0.41 () {54 vars in 918 bytes} [Fri May 27 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:03.189352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734cee-8914-4cce-8d13-2b75f3d95f2f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:03.189824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734cee-8914-4cce-8d13-2b75f3d95f2f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:03.189938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734cee-8914-4cce-8d13-2b75f3d95f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:03.208563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-6e734cee-8914-4cce-8d13-2b75f3d95f2f None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:03.209741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6e734cee-8914-4cce-8d13-2b75f3d95f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:03.219208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 707/1411] 10.222.0.41 () {58 vars in 1015 bytes} [Fri May 27 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:03.233103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560358d-b00a-45cf-a2fb-5554073b3589 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:03.233397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560358d-b00a-45cf-a2fb-5554073b3589 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:03.233687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560358d-b00a-45cf-a2fb-5554073b3589 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:03.235066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 705/1412] 10.222.0.41 () {54 vars in 918 bytes} [Fri May 27 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:03.239833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c416-e910-43f4-bbc9-0c05ef7dedcb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:03.240126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c416-e910-43f4-bbc9-0c05ef7dedcb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:03.240406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650c416-e910-43f4-bbc9-0c05ef7dedcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:03.258803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4650c416-e910-43f4-bbc9-0c05ef7dedcb None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:03.259990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4650c416-e910-43f4-bbc9-0c05ef7dedcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:03.269052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 708/1413] 10.222.0.41 () {58 vars in 1015 bytes} [Fri May 27 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:05.962807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7059-67b4-4324-ab9b-dc1d968868f3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:05.963377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7059-67b4-4324-ab9b-dc1d968868f3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:05.963377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a7059-67b4-4324-ab9b-dc1d968868f3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:05.964765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:05.970227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87d72-dd84-4d5f-8642-6343e1ea3673 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:05.970638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87d72-dd84-4d5f-8642-6343e1ea3673 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:05.970796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d87d72-dd84-4d5f-8642-6343e1ea3673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:05.988297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c5d87d72-dd84-4d5f-8642-6343e1ea3673 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:05.989551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c5d87d72-dd84-4d5f-8642-6343e1ea3673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:05.997956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:06.041875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7059-67b4-4324-ab9b-dc1d968868f3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:06.057397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a7059-67b4-4324-ab9b-dc1d968868f3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:06.062770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:06.063405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:06.063689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:06.063966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:06.072985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:06.074602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a21c8-0bb0-48cf-adef-394c00d58eac service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:06.083967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 707/1416] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:07.063639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c616-b2d3-4ad5-b6ba-db542921867a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:07.064168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c616-b2d3-4ad5-b6ba-db542921867a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:07.064290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7c616-b2d3-4ad5-b6ba-db542921867a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:07.065622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:07.071275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8401b6-bb59-401c-a197-85665b131996 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:07.071688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8401b6-bb59-401c-a197-85665b131996 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:07.071832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8401b6-bb59-401c-a197-85665b131996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:07.090379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ae8401b6-bb59-401c-a197-85665b131996 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:07.091578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ae8401b6-bb59-401c-a197-85665b131996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:07.099808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:07.271590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7c616-b2d3-4ad5-b6ba-db542921867a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:07.286364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7c616-b2d3-4ad5-b6ba-db542921867a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:07.292573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:07.293206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:07.293499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:07.293783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:07.301942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:07.303535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fc3ed-9212-41bd-b171-7c89877f317f service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:07.313253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 711/1419] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:08.245509 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56947f7-dd3f-4479-ab7f-cd96ab2147ba None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:08.246091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56947f7-dd3f-4479-ab7f-cd96ab2147ba None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:08.246091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56947f7-dd3f-4479-ab7f-cd96ab2147ba None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:08.247459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:08.252286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd1766-61f7-4d5a-af14-fbbdf7084a9c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:08.252761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd1766-61f7-4d5a-af14-fbbdf7084a9c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:08.252929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd1766-61f7-4d5a-af14-fbbdf7084a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:08.271563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-74cd1766-61f7-4d5a-af14-fbbdf7084a9c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:08.272971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-74cd1766-61f7-4d5a-af14-fbbdf7084a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:08.280964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:08.434007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56947f7-dd3f-4479-ab7f-cd96ab2147ba None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:08.449368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56947f7-dd3f-4479-ab7f-cd96ab2147ba None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:08.454774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:08.455411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:08.455695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:08.455973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:08.464756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:08.466330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696b40-fbc1-42f3-8f55-836c9ea95087 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:08.476251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 710/1422] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:09.394630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140436bd-7a6b-496a-8f7d-407970015839 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:09.395058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140436bd-7a6b-496a-8f7d-407970015839 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:09.395194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140436bd-7a6b-496a-8f7d-407970015839 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:09.396606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:09.402514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba119-c649-47f1-8c6a-2330b8068493 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:09.402944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba119-c649-47f1-8c6a-2330b8068493 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:09.403081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ba119-c649-47f1-8c6a-2330b8068493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:09.420900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-508ba119-c649-47f1-8c6a-2330b8068493 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:09.422102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-508ba119-c649-47f1-8c6a-2330b8068493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:09.430112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:09.580719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140436bd-7a6b-496a-8f7d-407970015839 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:09.595080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140436bd-7a6b-496a-8f7d-407970015839 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:09.600463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:09.601418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:09.601800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:09.602091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:09.610147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:09.611740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90abc3-d3af-4210-a661-2f1d947d2c63 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:09.621782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 714/1425] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:10.547481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced9211-55da-4edd-bc92-452464e2e1a6 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:10.548023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced9211-55da-4edd-bc92-452464e2e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:10.548023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced9211-55da-4edd-bc92-452464e2e1a6 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:10.549480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:10.555113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc06c8-5b40-414b-a598-5d7b0c41fad5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:10.555537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc06c8-5b40-414b-a598-5d7b0c41fad5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:10.555658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc06c8-5b40-414b-a598-5d7b0c41fad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:10.574202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4ecc06c8-5b40-414b-a598-5d7b0c41fad5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:10.575400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4ecc06c8-5b40-414b-a598-5d7b0c41fad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:10.583210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:10.734170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced9211-55da-4edd-bc92-452464e2e1a6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:10.749031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced9211-55da-4edd-bc92-452464e2e1a6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:10.754403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:10.755059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:10.755344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:10.755627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:10.763709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:10.765333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08003e-5481-42ac-946b-ee870693d19b service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:10.774711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 713/1428] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:11.640908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66918aeb-866d-427a-aae7-7ea189c84aa4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:11.641520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66918aeb-866d-427a-aae7-7ea189c84aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:11.641520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66918aeb-866d-427a-aae7-7ea189c84aa4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:11.642804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:11.648650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794024-026c-48d8-bed2-6d4d325ec6c3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:11.649232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794024-026c-48d8-bed2-6d4d325ec6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:11.649232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30794024-026c-48d8-bed2-6d4d325ec6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:11.667172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-30794024-026c-48d8-bed2-6d4d325ec6c3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:11.668363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-30794024-026c-48d8-bed2-6d4d325ec6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:11.676055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:11.826699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66918aeb-866d-427a-aae7-7ea189c84aa4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:11.841603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66918aeb-866d-427a-aae7-7ea189c84aa4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:11.847332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:11.847961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:11.848243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:11.848558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:11.857533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:11.859111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14e234-1709-4cb9-b3b4-8373353e08c0 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:11.868401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 717/1431] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:12.729302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0604e0-5c37-4640-8f31-58fa5b6b959b None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:12.729880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0604e0-5c37-4640-8f31-58fa5b6b959b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:12.729880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0604e0-5c37-4640-8f31-58fa5b6b959b None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:12.731216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:12.736061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0615-e7e1-4a56-88c9-35dced91c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:12.736517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0615-e7e1-4a56-88c9-35dced91c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:12.736661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d0615-e7e1-4a56-88c9-35dced91c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:12.755867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0d9d0615-e7e1-4a56-88c9-35dced91c9d0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:12.757678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0d9d0615-e7e1-4a56-88c9-35dced91c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:12.765543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:12.917151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0604e0-5c37-4640-8f31-58fa5b6b959b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:12.931444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0604e0-5c37-4640-8f31-58fa5b6b959b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:12.936764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f591188-4d74-4935-a432-c794151b267f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:12.937395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f591188-4d74-4935-a432-c794151b267f service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:12.937684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f591188-4d74-4935-a432-c794151b267f service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:12.937963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f591188-4d74-4935-a432-c794151b267f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:12.946889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f591188-4d74-4935-a432-c794151b267f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:12.948521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f591188-4d74-4935-a432-c794151b267f service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:12.958066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 716/1434] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:13.816171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e20f-0231-45d5-bf99-9007baf8cafe None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:13.816767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e20f-0231-45d5-bf99-9007baf8cafe None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:13.816767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233e20f-0231-45d5-bf99-9007baf8cafe None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:13.818155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:13.823317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9e9eb-25ae-4906-bf05-d61786ed4a2f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:13.823736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9e9eb-25ae-4906-bf05-d61786ed4a2f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:13.823893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9e9eb-25ae-4906-bf05-d61786ed4a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:13.842185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d6d9e9eb-25ae-4906-bf05-d61786ed4a2f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:13.843627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d6d9e9eb-25ae-4906-bf05-d61786ed4a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:13.852153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:14.003224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233e20f-0231-45d5-bf99-9007baf8cafe None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:14.018178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233e20f-0231-45d5-bf99-9007baf8cafe None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:14.024309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:14.024875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:14.025163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:14.025443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:14.033533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:14.035134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1479ad0-5a81-4ecc-b31c-27aa468bc996 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:14.044690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 720/1437] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:14.891574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f786f8-dc22-4200-bc3c-a4807b186afa None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:14.892122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f786f8-dc22-4200-bc3c-a4807b186afa None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:14.892122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f786f8-dc22-4200-bc3c-a4807b186afa None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:14.893550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:14.898610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374e9-dd06-4d9a-93b6-ca5854975ebb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:14.899045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374e9-dd06-4d9a-93b6-ca5854975ebb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:14.899202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99374e9-dd06-4d9a-93b6-ca5854975ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:14.917596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b99374e9-dd06-4d9a-93b6-ca5854975ebb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:14.918805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b99374e9-dd06-4d9a-93b6-ca5854975ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:14.926686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:15.076389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f786f8-dc22-4200-bc3c-a4807b186afa None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:15.090762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f786f8-dc22-4200-bc3c-a4807b186afa None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:15.096757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:15.097395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:15.097677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:15.097953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:15.108241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:15.110018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c44a7-8667-471f-a1cc-f412dacffa88 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:15.120125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 719/1440] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:17.215818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66983ea-0055-41f1-b551-b8148c46ffad None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:17.216395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66983ea-0055-41f1-b551-b8148c46ffad None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:17.216395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66983ea-0055-41f1-b551-b8148c46ffad None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:17.218047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:17.223307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4ab2-5083-4010-8dba-79532afce8a8 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:17.223738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4ab2-5083-4010-8dba-79532afce8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:17.223884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4ab2-5083-4010-8dba-79532afce8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:17.244334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-43ef4ab2-5083-4010-8dba-79532afce8a8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:17.245582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43ef4ab2-5083-4010-8dba-79532afce8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:17.255572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:17.294578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66983ea-0055-41f1-b551-b8148c46ffad None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:17.306705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66983ea-0055-41f1-b551-b8148c46ffad None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:17.310306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:17.310944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:17.311227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:17.311501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:17.321304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:17.322742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb339ee-4570-4478-a988-6e495c059a1d service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:17.332137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 723/1443] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:19.840040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:19.840595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:19.840691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:19.842044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:19.847644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd8d5c-32fc-4083-9c3e-38c56e3dd3cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:19.848170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd8d5c-32fc-4083-9c3e-38c56e3dd3cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:19.848270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd8d5c-32fc-4083-9c3e-38c56e3dd3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:19.866116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-25cd8d5c-32fc-4083-9c3e-38c56e3dd3cc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:19.867415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-25cd8d5c-32fc-4083-9c3e-38c56e3dd3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:19.875315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:05:19.915111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:19.916824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:19.929783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905966f6-e5c3-4fa6-8f23-af8d24afe631 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:19.933308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:19.933937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:19.934223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:19.934497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:19.943269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:19.945316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bb5f7-8e40-4fc0-831c-c0c512cb6967 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:19.954686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 722/1446] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:22.473561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde264-b71e-4ee1-893c-556fd664a150 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:22.474135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde264-b71e-4ee1-893c-556fd664a150 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:22.474135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde264-b71e-4ee1-893c-556fd664a150 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:22.475572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:22.481268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147298b5-36d2-40c2-899a-f40a622ac462 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:22.481675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147298b5-36d2-40c2-899a-f40a622ac462 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:22.481827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147298b5-36d2-40c2-899a-f40a622ac462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:22.500289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-147298b5-36d2-40c2-899a-f40a622ac462 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:22.500944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-147298b5-36d2-40c2-899a-f40a622ac462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:22.509885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:22.547577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde264-b71e-4ee1-893c-556fd664a150 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:22.558812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde264-b71e-4ee1-893c-556fd664a150 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:22.562570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:22.563211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:22.563496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:22.563775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:22.573758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:22.575221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fb52f-0ef3-4bb7-abd5-e901d29b9f78 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:22.584583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 726/1449] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:24.995742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9429c-c365-43a7-84f5-66a6af3ff9b3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:24.996358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9429c-c365-43a7-84f5-66a6af3ff9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:24.996358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9429c-c365-43a7-84f5-66a6af3ff9b3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:24.997707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:25.003225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c800-b066-4c8c-8b27-7fa65fd6a26e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:25.003663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c800-b066-4c8c-8b27-7fa65fd6a26e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:25.003792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c800-b066-4c8c-8b27-7fa65fd6a26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:25.022433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e5a7c800-b066-4c8c-8b27-7fa65fd6a26e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:25.023823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e5a7c800-b066-4c8c-8b27-7fa65fd6a26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:25.032677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:25.078501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9429c-c365-43a7-84f5-66a6af3ff9b3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:25.090102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9429c-c365-43a7-84f5-66a6af3ff9b3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:25.093868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:25.094504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:25.094789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:25.095067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:25.104258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:25.105727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195be468-8c46-4f97-b244-4061ecf34976 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:25.115707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 725/1452] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:27.399875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807c2e-9803-45e0-b93a-e9a30657fb5d None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:27.400338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807c2e-9803-45e0-b93a-e9a30657fb5d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:27.400532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3807c2e-9803-45e0-b93a-e9a30657fb5d None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:27.401921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:27.407370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f1d1-38e2-46a5-8a8b-8ef1dd0e5f3e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:27.407800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f1d1-38e2-46a5-8a8b-8ef1dd0e5f3e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:27.407931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6f1d1-38e2-46a5-8a8b-8ef1dd0e5f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:27.425826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fce6f1d1-38e2-46a5-8a8b-8ef1dd0e5f3e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:27.427012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fce6f1d1-38e2-46a5-8a8b-8ef1dd0e5f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:27.435578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:27.473809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3807c2e-9803-45e0-b93a-e9a30657fb5d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:27.486223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3807c2e-9803-45e0-b93a-e9a30657fb5d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:27.490075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:27.490713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:27.491066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:27.491881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:27.500794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:27.502236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81539d97-49ae-4530-8654-6497a59ff54e service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:27.512260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 729/1455] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:29.847962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee12bf-5b17-464c-88e0-9582cd382910 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:29.848598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee12bf-5b17-464c-88e0-9582cd382910 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:29.848598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee12bf-5b17-464c-88e0-9582cd382910 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:29.850007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:29.855578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380e90-0991-4bc7-941f-f189d4980efd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:29.856026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380e90-0991-4bc7-941f-f189d4980efd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:29.856130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380e90-0991-4bc7-941f-f189d4980efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:29.874262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7f380e90-0991-4bc7-941f-f189d4980efd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:29.875448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7f380e90-0991-4bc7-941f-f189d4980efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:29.884065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:29.922253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee12bf-5b17-464c-88e0-9582cd382910 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:29.933173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee12bf-5b17-464c-88e0-9582cd382910 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:29.936814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:29.937454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:29.937736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:29.938011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:29.947436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:29.948927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79463d14-316c-4903-a824-2c5abaaa92ec service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:29.958465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 728/1458] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:32.237888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749f0f-113c-4078-adba-4df77922bd87 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:32.238467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749f0f-113c-4078-adba-4df77922bd87 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:32.238467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749f0f-113c-4078-adba-4df77922bd87 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:32.239975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:32.245609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c1c6-2829-4dc3-8052-e3a3bb21534d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:32.246025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c1c6-2829-4dc3-8052-e3a3bb21534d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:32.246167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c1c6-2829-4dc3-8052-e3a3bb21534d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:32.252140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=70 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:32.265346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d910c1c6-2829-4dc3-8052-e3a3bb21534d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:32.266532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d910c1c6-2829-4dc3-8052-e3a3bb21534d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:32.275286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:32.312826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2749f0f-113c-4078-adba-4df77922bd87 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:32.318287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=73 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:32.324892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2749f0f-113c-4078-adba-4df77922bd87 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:32.328400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:32.329062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:32.329347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:32.329630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:32.339250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:32.340728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef20712-8954-4982-a005-d9bfc6037fa0 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:32.351103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 732/1461] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:34.612184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51264-00a1-4c44-80e7-80148c086018 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:34.612781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51264-00a1-4c44-80e7-80148c086018 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:34.612781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe51264-00a1-4c44-80e7-80148c086018 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:34.614172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:34.619636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cafa-0f1c-4f47-a95d-55d8014c7a8c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:34.620086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cafa-0f1c-4f47-a95d-55d8014c7a8c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:34.620192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5cafa-0f1c-4f47-a95d-55d8014c7a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:34.640156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-74d5cafa-0f1c-4f47-a95d-55d8014c7a8c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:34.641318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-74d5cafa-0f1c-4f47-a95d-55d8014c7a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:34.650000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:34.688694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe51264-00a1-4c44-80e7-80148c086018 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:34.700063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe51264-00a1-4c44-80e7-80148c086018 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:34.703475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:34.704101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:34.704381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:34.704687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:34.714026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:34.715452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e129ba6-db41-4dc7-85c4-c634a91d0e9e service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:34.725064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 731/1464] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:36.971258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f3d9-1a37-432d-8e82-9e7839ef7389 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:36.971815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f3d9-1a37-432d-8e82-9e7839ef7389 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:36.971920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f3d9-1a37-432d-8e82-9e7839ef7389 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:36.973352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:36.979488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abb326-6c03-42b6-8949-8f34ee638a99 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:36.979942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abb326-6c03-42b6-8949-8f34ee638a99 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:36.980061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abb326-6c03-42b6-8949-8f34ee638a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:36.999250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-86abb326-6c03-42b6-8949-8f34ee638a99 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:37.000574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-86abb326-6c03-42b6-8949-8f34ee638a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:37.009578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:05:37.047732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f3d9-1a37-432d-8e82-9e7839ef7389 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:37.058818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f3d9-1a37-432d-8e82-9e7839ef7389 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:37.062411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:37.063049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:37.063334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:37.063615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:37.073206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:37.074666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a79942-cd60-4515-97d9-412659532900 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:37.084009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 735/1467] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:39.321110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035eda8-6b09-49e9-be69-17c36edd0d71 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:39.321636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035eda8-6b09-49e9-be69-17c36edd0d71 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:39.321766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035eda8-6b09-49e9-be69-17c36edd0d71 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:39.323081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:05:39.328393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e76a-6d13-4bbf-9166-1adcb3a8a02c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:39.328860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e76a-6d13-4bbf-9166-1adcb3a8a02c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:39.329038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e76a-6d13-4bbf-9166-1adcb3a8a02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:39.348406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cb74e76a-6d13-4bbf-9166-1adcb3a8a02c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:39.349636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cb74e76a-6d13-4bbf-9166-1adcb3a8a02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:39.357765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:39.394635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035eda8-6b09-49e9-be69-17c36edd0d71 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:39.406028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035eda8-6b09-49e9-be69-17c36edd0d71 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:39.409534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:39.410166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:39.410451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:39.410724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:39.419692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:39.421151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254c213-2c3b-4a8e-9cb7-2a56385d714a service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:39.430485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 734/1470] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:05:49.342867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=17 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:49.426287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=17 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:12.778491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b70137-db7e-4889-95cd-b7c8963e86cc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:12.779050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b70137-db7e-4889-95cd-b7c8963e86cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:12.779050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b70137-db7e-4889-95cd-b7c8963e86cc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:12.780358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 737/1471] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:12.785783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9d18-0fee-4f7c-ac78-b5fc67f7cc44 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:12.786191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9d18-0fee-4f7c-ac78-b5fc67f7cc44 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:12.786344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c9d18-0fee-4f7c-ac78-b5fc67f7cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:12.804536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-071c9d18-0fee-4f7c-ac78-b5fc67f7cc44 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:12.805734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-071c9d18-0fee-4f7c-ac78-b5fc67f7cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:12.813426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 735/1472] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:12.861140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09129-0e18-4202-b99c-17ab078efb4c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:12.861434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09129-0e18-4202-b99c-17ab078efb4c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:12.861719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b09129-0e18-4202-b99c-17ab078efb4c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:12.863118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 738/1473] 10.222.0.41 () {54 vars in 962 bytes} [Fri May 27 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:12.868307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03971e-a4d4-4e06-9b5e-48e3ae31f586 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:12.868629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03971e-a4d4-4e06-9b5e-48e3ae31f586 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:12.868910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03971e-a4d4-4e06-9b5e-48e3ae31f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:12.887035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-af03971e-a4d4-4e06-9b5e-48e3ae31f586 None None] MFA Rules not processed for user `36672cdeb2f148698b116e06095955d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:12.888207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-af03971e-a4d4-4e06-9b5e-48e3ae31f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:12.894515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 736/1474] 10.222.0.41 () {58 vars in 1059 bytes} [Fri May 27 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:12.902456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b09129-0e18-4202-b99c-17ab078efb4c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:12.913774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b09129-0e18-4202-b99c-17ab078efb4c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:12.917327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:12.917958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:12.918246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:12.918526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:12.927342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:12.928919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0898ea6-3cf6-4879-b9f3-12941f79eca9 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:12.938245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 739/1475] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:15.237835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801e85-93b1-45a2-86be-6763299150f3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:15.238396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801e85-93b1-45a2-86be-6763299150f3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:15.238396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf801e85-93b1-45a2-86be-6763299150f3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:15.239771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 737/1476] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:15.245102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff6c17-38c0-4dbf-95c2-4a68d89e99d2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:15.245522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff6c17-38c0-4dbf-95c2-4a68d89e99d2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:15.245693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff6c17-38c0-4dbf-95c2-4a68d89e99d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:15.264054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7eff6c17-38c0-4dbf-95c2-4a68d89e99d2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:15.265294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7eff6c17-38c0-4dbf-95c2-4a68d89e99d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:15.274391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 740/1477] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:15.296954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf801e85-93b1-45a2-86be-6763299150f3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:15.309233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf801e85-93b1-45a2-86be-6763299150f3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:15.313602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:15.314242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:15.314529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:15.314805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:15.315533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:15.317359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:06:15.317359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:06:15.317359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:06:15.319045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e2b8d-8df4-4fe1-bb57-979ed0de94b5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:15.338584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 738/1478] 10.222.0.41 () {56 vars in 1108 bytes} [Fri May 27 15:06:15 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:06:17.569520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c692-de84-4392-a376-5aa3963e0efb None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:17.570074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c692-de84-4392-a376-5aa3963e0efb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:17.570074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c692-de84-4392-a376-5aa3963e0efb None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:17.571528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 741/1479] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:17.577020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd750f-2edd-4c39-afe4-00e6f7b36b1d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:17.577463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd750f-2edd-4c39-afe4-00e6f7b36b1d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:17.577575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd750f-2edd-4c39-afe4-00e6f7b36b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:17.595940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f6dd750f-2edd-4c39-afe4-00e6f7b36b1d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:17.597152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f6dd750f-2edd-4c39-afe4-00e6f7b36b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:17.605200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 739/1480] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:06:17.649175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c692-de84-4392-a376-5aa3963e0efb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:17.663646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c692-de84-4392-a376-5aa3963e0efb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:17.668968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:17.669597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:17.669880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:17.670156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:17.678355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:17.680061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd15269-ff06-42b1-abbe-e4fe29d11ee5 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:17.689270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 742/1481] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:20.082077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7698ac-af0a-4cef-80d4-1b2fc3706073 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:20.082636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7698ac-af0a-4cef-80d4-1b2fc3706073 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:20.082636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7698ac-af0a-4cef-80d4-1b2fc3706073 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:20.084077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 740/1482] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:20.089460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d335b4-f662-467f-be03-1fd2fd468220 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:20.089875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d335b4-f662-467f-be03-1fd2fd468220 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:20.089998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d335b4-f662-467f-be03-1fd2fd468220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:20.108135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e5d335b4-f662-467f-be03-1fd2fd468220 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:20.109377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e5d335b4-f662-467f-be03-1fd2fd468220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:20.117722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 743/1483] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:20.155287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7698ac-af0a-4cef-80d4-1b2fc3706073 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:20.166161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7698ac-af0a-4cef-80d4-1b2fc3706073 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:20.169645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:20.170276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:20.170559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:20.170835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:20.180322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:20.181808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5a451-cce6-40f9-9b25-3b990251f20a service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:20.191313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 741/1484] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:22.499735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334f9b-176b-4bb4-91c6-9062678da8c7 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.500291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334f9b-176b-4bb4-91c6-9062678da8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.500291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334f9b-176b-4bb4-91c6-9062678da8c7 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.501715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 744/1485] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:22.507711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e16-d00a-4350-adda-51cba3391ade None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.508128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e16-d00a-4350-adda-51cba3391ade None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.508305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0e16-d00a-4350-adda-51cba3391ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.526309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-719f0e16-d00a-4350-adda-51cba3391ade None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:22.527512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-719f0e16-d00a-4350-adda-51cba3391ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:22.535270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 742/1486] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:22.573450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e334f9b-176b-4bb4-91c6-9062678da8c7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:22.584202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e334f9b-176b-4bb4-91c6-9062678da8c7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:22.587760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:22.588392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.588714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.588996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.598303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:22.599892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0b70e-1c56-4dba-8b86-30a6b0f80860 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:22.609417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 745/1487] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:24.940752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347a44-b053-46db-8893-8d1cccb64929 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:24.941321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347a44-b053-46db-8893-8d1cccb64929 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:24.941321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45347a44-b053-46db-8893-8d1cccb64929 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:24.942756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 743/1488] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:24.948866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b809-2f06-487c-8ad3-36f20300154b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:24.949279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b809-2f06-487c-8ad3-36f20300154b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:24.949423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95b809-2f06-487c-8ad3-36f20300154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:24.967939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7e95b809-2f06-487c-8ad3-36f20300154b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:24.969399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7e95b809-2f06-487c-8ad3-36f20300154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:24.977648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 746/1489] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:25.017611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45347a44-b053-46db-8893-8d1cccb64929 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:25.032886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45347a44-b053-46db-8893-8d1cccb64929 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:25.036350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:25.037007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:25.037811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:25.038102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:25.050149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:25.051803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be002-041a-4f7f-a4fc-3d80625afdd6 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:25.063585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 744/1490] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:06:27.492324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb542e8-b9b3-4760-b7f4-702b6ba3403a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:27.492910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb542e8-b9b3-4760-b7f4-702b6ba3403a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:27.493009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb542e8-b9b3-4760-b7f4-702b6ba3403a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:27.494336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 747/1491] 10.222.0.41 () {54 vars in 926 bytes} [Fri May 27 15:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:27.499611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677ec0-cea5-449d-a28a-c54fd432fdab None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:27.500016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677ec0-cea5-449d-a28a-c54fd432fdab None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:27.500174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677ec0-cea5-449d-a28a-c54fd432fdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:27.518444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7e677ec0-cea5-449d-a28a-c54fd432fdab None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:27.519624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7e677ec0-cea5-449d-a28a-c54fd432fdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:27.528382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 745/1492] 10.222.0.41 () {58 vars in 1023 bytes} [Fri May 27 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:27.566743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb542e8-b9b3-4760-b7f4-702b6ba3403a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:27.578289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb542e8-b9b3-4760-b7f4-702b6ba3403a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:27.581838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:27.582468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:27.582756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:27.583040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:27.592219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:27.593961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe83e1-134d-497e-928a-f19ae17d0a05 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:27.603586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 748/1493] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:37.513389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=33 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:37.599178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=33 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:10.132245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.132875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.132875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.151230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:10.152418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:10.160756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 746/1494] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:10.169945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.170457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.170560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.188981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:10.190173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:10.198161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 749/1495] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:10.202685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.212667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ebdca-4a77-406d-9294-6ca7b90f93aa None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.217117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.217766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.218051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.218342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.219320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:08:10.219627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.220677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.220677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:08:10.220677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:08:10.221579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.233555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 747/1496] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:08:10.242185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.251482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85f82b-588f-4bd5-aba9-b483fd597b5d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.256132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.256810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.257094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.257370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.258069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.259853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.259853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:08:10.259853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:08:10.261107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.301270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 750/1497] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:08:10.303989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.313930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c837b53-ede6-4174-9641-d8627c00569c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.371366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.372034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.372337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.372632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.373324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.374343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.374343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:08:10.374343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:08:10.375568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.396345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 748/1498] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:08:10.400478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.411226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c9f59-a2c2-4bd4-8205-fbefa0ef72d0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.415845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.416525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.416810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.417088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.417800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.418824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.418824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:08:10.418824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:08:10.419698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.432419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 751/1499] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:08:10.436560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.446492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6822e3c-4fc3-4dd1-8f3e-d49baa29544d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.450946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.451598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.451887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.452185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.452958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.454004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.454004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:08:10.454004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:08:10.454885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af526f0-e5a9-4a55-9a26-894b7386eb28 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.468823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 749/1500] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:08:10.473894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.483890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c71dc-5ab6-4a1c-a8f0-7398716b3f4d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.488367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.489260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.489541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.489821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] PATH_INFO: `/v3/projects/8f551b8c020e4407a7e435c631374121/users/514cfaea5eea41d2a502d1cc16d54044/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.505580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f551b8c020e4407a7e435c631374121, user_id=514cfaea5eea41d2a502d1cc16d54044, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.506634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.506634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:08:10.506634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:08:10.508604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1211323-5371-493b-a516-c20f85d6239d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.521885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 752/1501] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:08:10 2022] PUT /identity/v3/projects/8f551b8c020e4407a7e435c631374121/users/514cfaea5eea41d2a502d1cc16d54044/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:08:10.533162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab07a-9f48-44c1-9cb7-9d51e3add44a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.533459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab07a-9f48-44c1-9cb7-9d51e3add44a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.533739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab07a-9f48-44c1-9cb7-9d51e3add44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.559997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2deab07a-9f48-44c1-9cb7-9d51e3add44a None None] MFA Rules not processed for user `514cfaea5eea41d2a502d1cc16d54044`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:10.561423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2deab07a-9f48-44c1-9cb7-9d51e3add44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:10.626746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 750/1502] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:10.639748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d6ea0-cbc3-4a52-ad41-a192776e8816 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.639987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d6ea0-cbc3-4a52-ad41-a192776e8816 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.640268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d6ea0-cbc3-4a52-ad41-a192776e8816 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.641672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 753/1503] 10.222.0.41 () {54 vars in 963 bytes} [Fri May 27 15:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:10.646925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4c3b-3c1c-46ee-9616-107714e0d618 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.647223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4c3b-3c1c-46ee-9616-107714e0d618 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.647508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f4c3b-3c1c-46ee-9616-107714e0d618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.665335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4a8f4c3b-3c1c-46ee-9616-107714e0d618 None None] MFA Rules not processed for user `c1a532f2e8b446b7a03b38b3dda377a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:10.666525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4a8f4c3b-3c1c-46ee-9616-107714e0d618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:10.710746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:10.719479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 751/1504] 10.222.0.41 () {58 vars in 1060 bytes} [Fri May 27 15:08:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:10.829954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d6ea0-cbc3-4a52-ad41-a192776e8816 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.841407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d6ea0-cbc3-4a52-ad41-a192776e8816 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.845114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.845735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.846012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.846283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.854605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.856027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:10.864964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 754/1505] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:08:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:10.907603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f95e85-31c1-4780-ad45-34e0f334bbf0 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.907895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f95e85-31c1-4780-ad45-34e0f334bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.908176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f95e85-31c1-4780-ad45-34e0f334bbf0 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.910097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 752/1506] 10.222.0.41 () {52 vars in 829 bytes} [Fri May 27 15:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:10.913804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:10.975334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef74730-9d4c-4757-b1b3-cddd73396a7b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:10.980097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:10.980996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:10.981283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:10.981566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] PATH_INFO: `/v3/users/514cfaea5eea41d2a502d1cc16d54044` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:10.982755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=514cfaea5eea41d2a502d1cc16d54044)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:10.983788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:10.983788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:08:10.983788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:08:10.985039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:11.051064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '514cfaea5eea41d2a502d1cc16d54044'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.060629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '514cfaea5eea41d2a502d1cc16d54044'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.061324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '514cfaea5eea41d2a502d1cc16d54044'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.079590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '514cfaea5eea41d2a502d1cc16d54044'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.080237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '514cfaea5eea41d2a502d1cc16d54044'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.088848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:11.100562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 755/1507] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:08:10 2022] DELETE /identity/v3/users/514cfaea5eea41d2a502d1cc16d54044 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:08:11.104934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:11.105229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:11.105516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:11.123334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:11.124591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:11.183036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 753/1508] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:11.193403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:11.194838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:11.250801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d04ee-f308-462d-8102-21649a395531 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:11.255199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:11.255848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:11.256133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:11.256415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:11.265333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:11.266784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce245c33-8db6-45ec-90fc-97a2437db347 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:11.275969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 756/1509] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:11.610109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:11.611542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:11.623726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d43e3-e89e-4759-921f-f4c73967835c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:11.628948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:11.629821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:11.630103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:11.630380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] PATH_INFO: `/v3/projects/8f551b8c020e4407a7e435c631374121` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:11.632223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f551b8c020e4407a7e435c631374121)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:11.632811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:11.632811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:08:11.632811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:08:11.634191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:11.696359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f551b8c020e4407a7e435c631374121', 'request_id': 'req-8b23679b-4e7c-40c8-829b-e9a631c334ef'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.709725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f551b8c020e4407a7e435c631374121', 'request_id': 'req-8b23679b-4e7c-40c8-829b-e9a631c334ef'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.715830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f551b8c020e4407a7e435c631374121', 'request_id': 'req-8b23679b-4e7c-40c8-829b-e9a631c334ef'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.731129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] The token cache is being invalidate because project 8f551b8c020e4407a7e435c631374121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:08:11.731418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.732121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8b23679b-4e7c-40c8-829b-e9a631c334ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:08:11.733711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 754/1510] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:08:11 2022] DELETE /identity/v3/projects/8f551b8c020e4407a7e435c631374121 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:08:21.271933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=29 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:21.728087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:09:55.076656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:55.077244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:55.077244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:55.078603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:55.083668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325900c-f2da-4420-b350-9d4fe760ddf7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:55.084143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325900c-f2da-4420-b350-9d4fe760ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:55.084239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325900c-f2da-4420-b350-9d4fe760ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:55.103381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4325900c-f2da-4420-b350-9d4fe760ddf7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:55.104641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4325900c-f2da-4420-b350-9d4fe760ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:55.165409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:55.337309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:55.339010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:55.416461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97dd75-7737-45ee-81d9-83beaadb90cc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:55.421896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:55.422543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:55.422830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:55.423127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:55.431177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:55.432829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145f07c-85c0-4558-b76c-9f94db0dd510 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:55.442488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 758/1513] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:55.737437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21bb7-0bb6-4ef9-967a-60630cd0acdf None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:55.737640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21bb7-0bb6-4ef9-967a-60630cd0acdf None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:55.737912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21bb7-0bb6-4ef9-967a-60630cd0acdf None None] PATH_INFO: `/v3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:55.739231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 756/1514] 10.222.0.40 () {54 vars in 925 bytes} [Fri May 27 15:09:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:09:55.745946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3557-2870-4629-be6b-e61ce0a3b357 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:55.746278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3557-2870-4629-be6b-e61ce0a3b357 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:55.746585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c3557-2870-4629-be6b-e61ce0a3b357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:55.766100 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4e2c3557-2870-4629-be6b-e61ce0a3b357 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:55.767344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4e2c3557-2870-4629-be6b-e61ce0a3b357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:55.830867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 759/1515] 10.222.0.40 () {58 vars in 1015 bytes} [Fri May 27 15:09:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:56.329058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555aa5b-514d-4140-a09a-3f1e3ec5d0ec None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:56.329602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555aa5b-514d-4140-a09a-3f1e3ec5d0ec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:56.329602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555aa5b-514d-4140-a09a-3f1e3ec5d0ec None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:56.331011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 757/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:56.335781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45812-a605-4c17-a0b1-811524eb240d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:56.336199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45812-a605-4c17-a0b1-811524eb240d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:56.336353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45812-a605-4c17-a0b1-811524eb240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:56.355176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b4e45812-a605-4c17-a0b1-811524eb240d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:56.356375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4e45812-a605-4c17-a0b1-811524eb240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:56.364333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 760/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:56.517265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555aa5b-514d-4140-a09a-3f1e3ec5d0ec None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:56.532048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555aa5b-514d-4140-a09a-3f1e3ec5d0ec None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:56.537363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:56.537989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:56.538270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:56.538545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:56.547256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:56.548964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73c56e-7a4e-490f-a2c6-140b100faa7a service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:56.558128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 758/1518] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:09:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:59.625012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdfda-837d-48c5-b3d3-f373e6a86448 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:59.625572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdfda-837d-48c5-b3d3-f373e6a86448 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:59.625572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fdfda-837d-48c5-b3d3-f373e6a86448 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:59.627456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 761/1519] 10.222.0.41 () {54 vars in 960 bytes} [Fri May 27 15:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:59.632622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e22e1-b939-42e6-8671-496366d57c8c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:59.633104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e22e1-b939-42e6-8671-496366d57c8c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:59.633802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e22e1-b939-42e6-8671-496366d57c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:59.654783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fb5e22e1-b939-42e6-8671-496366d57c8c None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:59.655293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fb5e22e1-b939-42e6-8671-496366d57c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:59.663999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 759/1520] 10.222.0.41 () {58 vars in 1057 bytes} [Fri May 27 15:09:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:59.671548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fdfda-837d-48c5-b3d3-f373e6a86448 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:59.687134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fdfda-837d-48c5-b3d3-f373e6a86448 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:59.692783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:59.693611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:59.694013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:59.694417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:59.705517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:59.707877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e14f1-a13a-47cd-96b4-4684f178f070 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:59.721277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 762/1521] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:09:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:10:07.846987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0144b8d-228d-4d8b-a276-a733a4b822c3 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:07.847577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0144b8d-228d-4d8b-a276-a733a4b822c3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:07.847577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0144b8d-228d-4d8b-a276-a733a4b822c3 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:07.848994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:07.853778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294a07-45eb-48d3-a835-27714d2de615 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:07.854236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294a07-45eb-48d3-a835-27714d2de615 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:07.854346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294a07-45eb-48d3-a835-27714d2de615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:07.874787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-da294a07-45eb-48d3-a835-27714d2de615 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:07.875996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-da294a07-45eb-48d3-a835-27714d2de615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:07.883751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:08.039241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0144b8d-228d-4d8b-a276-a733a4b822c3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:08.054382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0144b8d-228d-4d8b-a276-a733a4b822c3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:08.060274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:08.061263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:08.061550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:08.061831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:08.070559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:08.072138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace26be-a1b3-4024-8f95-d5b8f6715fce service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:08.081844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 761/1524] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:08.953703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7433e-c02b-45e6-886e-41c38394f310 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:08.954286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7433e-c02b-45e6-886e-41c38394f310 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:08.954286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7433e-c02b-45e6-886e-41c38394f310 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:08.955652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:08.960543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1537e77-aca0-4ca6-9487-8135fe244425 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:08.960972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1537e77-aca0-4ca6-9487-8135fe244425 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:08.961129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1537e77-aca0-4ca6-9487-8135fe244425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:08.979993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c1537e77-aca0-4ca6-9487-8135fe244425 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:08.981904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c1537e77-aca0-4ca6-9487-8135fe244425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:08.989579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:09.140963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7433e-c02b-45e6-886e-41c38394f310 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:09.155623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7433e-c02b-45e6-886e-41c38394f310 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:09.161080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:09.161734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:09.162066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:09.162477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:09.173569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:09.175181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3f81-a847-46c3-acc9-fae37a07686a service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:09.184770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 765/1527] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:13.375947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:13.376536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:13.376536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:13.377902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:13.383267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065c732-7481-44b2-b5b4-a849ede903b9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:13.383725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065c732-7481-44b2-b5b4-a849ede903b9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:13.383829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065c732-7481-44b2-b5b4-a849ede903b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:13.401910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d065c732-7481-44b2-b5b4-a849ede903b9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:13.403121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d065c732-7481-44b2-b5b4-a849ede903b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:13.411034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:13.446887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:13.448593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:13.505034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a66e3-cbb8-453b-a8e8-9ae3e92b2c00 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:13.508543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:13.509179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:13.509459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:13.509731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:13.518893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:13.520587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808b4e-7298-4248-bf81-5e5a912b6316 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:13.531038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 764/1530] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:15.758258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541494c-e639-41c7-add2-5d2925218a30 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:15.758786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541494c-e639-41c7-add2-5d2925218a30 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:15.758786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541494c-e639-41c7-add2-5d2925218a30 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:15.760133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:10:15.765964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526866f-206a-49bf-9b5c-7ee1d6f20775 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:15.766379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526866f-206a-49bf-9b5c-7ee1d6f20775 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:15.766523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526866f-206a-49bf-9b5c-7ee1d6f20775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:15.785373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b526866f-206a-49bf-9b5c-7ee1d6f20775 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:15.786600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b526866f-206a-49bf-9b5c-7ee1d6f20775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:15.794771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:15.831264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541494c-e639-41c7-add2-5d2925218a30 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:15.843001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541494c-e639-41c7-add2-5d2925218a30 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:15.846583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:15.847216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:15.847499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:15.847777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:15.856793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:15.858241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d4d3f-9e74-47b3-9241-71f324b61f8f service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:15.867566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 768/1533] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:19.793204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:19.793775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:19.793938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:19.795385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:19.800916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322c01-0178-4a3d-8120-83314cdf96ea None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:19.801385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322c01-0178-4a3d-8120-83314cdf96ea None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:19.801599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4322c01-0178-4a3d-8120-83314cdf96ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:19.820052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b4322c01-0178-4a3d-8120-83314cdf96ea None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:19.821282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4322c01-0178-4a3d-8120-83314cdf96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:19.829834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:20.102342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:20.104045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:20.161820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81483eb0-0867-44fc-a5ea-4e56bbbb17c1 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:20.165380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:20.166024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:20.166309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:20.166588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:20.176875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:20.178309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39faa6-9d49-4d2a-910b-d093cff953e6 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:20.183270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:20.189926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 767/1536] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:22.441305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:22.441828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:22.441955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:22.443248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:22.448614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5020a-acb1-4e1c-ac34-c8d595254c94 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:22.449191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5020a-acb1-4e1c-ac34-c8d595254c94 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:22.450051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5020a-acb1-4e1c-ac34-c8d595254c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:22.468687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-07f5020a-acb1-4e1c-ac34-c8d595254c94 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:22.469870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-07f5020a-acb1-4e1c-ac34-c8d595254c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:22.477562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:22.751120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:22.752856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:22.760423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:22.767390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f563e-f27c-45b3-85e9-9fafa85b3f40 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:22.770963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361324c-d117-435b-8e12-f406637353de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:22.771600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361324c-d117-435b-8e12-f406637353de service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:22.771885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361324c-d117-435b-8e12-f406637353de service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:22.772160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361324c-d117-435b-8e12-f406637353de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:22.781483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361324c-d117-435b-8e12-f406637353de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:22.782944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361324c-d117-435b-8e12-f406637353de service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:22.792242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 771/1539] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:32.462551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:32.788129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:53.431644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.432311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.432418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.461958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.463756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.475402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 769/1540] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:53.487258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586e41-8098-4465-9f95-24e961f092a3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.487862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586e41-8098-4465-9f95-24e961f092a3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.487862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa586e41-8098-4465-9f95-24e961f092a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.515244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fa586e41-8098-4465-9f95-24e961f092a3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.516534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fa586e41-8098-4465-9f95-24e961f092a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.531522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 772/1541] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:53.537280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:53.554577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.554947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.555154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.557274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288014-69ca-4a94-8c7e-ac1c3dbcc4b9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:53.565934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:53.567052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.567564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.568065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.569077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:53.569636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:53.572851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d66b9f-e7a2-41b6-8ed2-213e34fd0850 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:53.585785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.589055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.603236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 770/1542] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:53.611051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 773/1543] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:11:53.621654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.622534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.623099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.630120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:53.644303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd7f21-8675-41ed-b0f6-3535d3d36a83 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:53.652808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.654335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.668758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 771/1544] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:53.672553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:53.673250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.673712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.674003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.674373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:53.674716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:53.679590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:53.685975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629ede-001c-458f-8d73-c69c1818476d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:53.693446 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:53.694430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.694934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.695421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.696646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:53.697232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:53.699090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:53.723162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 772/1545] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:11:53.727785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 774/1546] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:53.731725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:53.732805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:53.747703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e82c7-6df5-4b32-94bc-c462d21a847d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:53.845894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4dcb8-d431-44bb-93fc-078cc2595a27 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:53.856064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:53.856799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.857091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.858857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.859569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:53.860671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:11:53.860671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:11:53.860671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:11:53.864940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b70aa-1586-487a-843d-cc472b603148 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:53.869557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:53.870261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.870551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.870834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.871743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:53.872995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:11:53.872995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:11:53.872995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:11:53.874452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc38b89-6ab6-4733-8c13-cfd7c7ffb984 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:53.895169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 775/1547] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:11:53.896146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167840aa-d229-490e-9e91-2b1b6e0e769c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.896477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167840aa-d229-490e-9e91-2b1b6e0e769c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.896768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167840aa-d229-490e-9e91-2b1b6e0e769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.918547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 773/1548] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:53.920089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7daccb-4cd7-4341-a6cd-561a28a8cbb0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.920466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7daccb-4cd7-4341-a6cd-561a28a8cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.920875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7daccb-4cd7-4341-a6cd-561a28a8cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:53.921922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-167840aa-d229-490e-9e91-2b1b6e0e769c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.923314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-167840aa-d229-490e-9e91-2b1b6e0e769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.943183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-cb7daccb-4cd7-4341-a6cd-561a28a8cbb0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:53.944807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-cb7daccb-4cd7-4341-a6cd-561a28a8cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:53.989721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 776/1549] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:53.990797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:53.991080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:53.991369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.011864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.013121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.017045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 774/1550] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.021785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 777/1551] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.022315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935fb2-4ee7-4e98-a493-c615637306d9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.028473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.028647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935fb2-4ee7-4e98-a493-c615637306d9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.028647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d935fb2-4ee7-4e98-a493-c615637306d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.033394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4acdd9-9157-4fea-a8fe-5262344bb79c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.038017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.038713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.039003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.039286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.040657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.042534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.046016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6d935fb2-4ee7-4e98-a493-c615637306d9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.047221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6d935fb2-4ee7-4e98-a493-c615637306d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.052869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 778/1552] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:54.053703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.055363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 775/1553] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.056911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50dc0d-39f1-4d4b-8004-01d7c0dc58cd None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.057201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50dc0d-39f1-4d4b-8004-01d7c0dc58cd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.057481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50dc0d-39f1-4d4b-8004-01d7c0dc58cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.070111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35261c-ccb5-4155-90cf-d38a52046801 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.074721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.075403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.075547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1b50dc0d-39f1-4d4b-8004-01d7c0dc58cd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.075708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.075962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.076680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.076782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1b50dc0d-39f1-4d4b-8004-01d7c0dc58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.084621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 776/1554] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.088579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f63f3-3d9b-4956-9a74-397336241ae8 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.088579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f63f3-3d9b-4956-9a74-397336241ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.088579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f63f3-3d9b-4956-9a74-397336241ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.089322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f60e3-c110-42a4-b1de-3f5af9bd04e4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.110699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ba1f63f3-3d9b-4956-9a74-397336241ae8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.111923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ba1f63f3-3d9b-4956-9a74-397336241ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.115422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 779/1555] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/users => generated 286 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.115422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55f4b8-83d0-4d61-953b-3ed11852ff1a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.115422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55f4b8-83d0-4d61-953b-3ed11852ff1a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.115422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55f4b8-83d0-4d61-953b-3ed11852ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.123390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 777/1556] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.124952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.125241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.125519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.133745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cd55f4b8-83d0-4d61-953b-3ed11852ff1a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.134961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cd55f4b8-83d0-4d61-953b-3ed11852ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.144423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 780/1557] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.144601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.144715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f024-7d90-481f-9d12-3d200f017970 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.144807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.144892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f024-7d90-481f-9d12-3d200f017970 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.144892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8f024-7d90-481f-9d12-3d200f017970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.154708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 778/1558] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.155859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.169109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c68466-38cd-46ed-8c92-028aaf873e26 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.173774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.174012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-85b8f024-7d90-481f-9d12-3d200f017970 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.174437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.174720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.175127 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.175235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-85b8f024-7d90-481f-9d12-3d200f017970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.175876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.177554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bed42-6f33-4c7c-91d8-246fdc85ff37 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.187146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 781/1559] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.190998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.190998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.190998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.191265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 779/1560] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:54.191265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.191265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.191265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.205823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.213355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.213676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.214581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.215003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.222621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 780/1561] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.223365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 782/1562] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.223488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.225241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.228145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.234561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8cc12-5b23-49aa-be0d-2c7035980de0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.237368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d86039-f7c6-4ed9-8725-d830800e9986 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.239476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.240137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.240421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.240729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.241455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.241886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.242542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.242866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.243140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.243261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf445f5f-f6de-4503-b364-cd2db59154ce admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.243861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.244167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.245236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:11:54.245236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:11:54.245236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:11:54.246121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.253750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 781/1563] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:54.254997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd4635-24fc-452e-8433-879433b13657 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.255293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd4635-24fc-452e-8433-879433b13657 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.255574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd4635-24fc-452e-8433-879433b13657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.260857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 783/1564] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.261750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.273988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-62cd4635-24fc-452e-8433-879433b13657 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.275749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-62cd4635-24fc-452e-8433-879433b13657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.278892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa62f1-e890-466f-a9a9-8722455c9b2c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.283371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.283848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 782/1565] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.284135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.284519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.284705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd4635-24fc-452e-8433-879433b13657 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.284806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.285540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.285849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.287621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.294270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd4635-24fc-452e-8433-879433b13657 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.298838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.299481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.299762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.300038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.300798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.301005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 784/1566] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.301115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.301830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.302697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.313002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db9fd-e742-41ab-bbe6-610728c51fbf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.315167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 783/1567] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.315876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.322144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.323039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.323326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.323603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] PATH_INFO: `/v3/projects/c162124048a34243867e236c81a70eb0/users/d25dc63e8a4c4af8af2fe29b7819212b/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.329496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33f8e7-f414-4d4e-99a4-8fe8267be2c0 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.333947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.334586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.334872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.335148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.335857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.336268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.338363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.338766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c162124048a34243867e236c81a70eb0, user_id=d25dc63e8a4c4af8af2fe29b7819212b, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.341756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.351356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 784/1568] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.358772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.362250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d264b-5422-46e3-a463-fedccf2c48eb admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.363776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 785/1569] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:54 2022] PUT /identity/v3/projects/c162124048a34243867e236c81a70eb0/users/d25dc63e8a4c4af8af2fe29b7819212b/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:54.364627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.427790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.428370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.428690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.428997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.429717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.430049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.430713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b59780-4661-40bd-aab3-bf21b4c33807 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.432033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.436161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.436860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.437140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.437410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.438137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.439726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.443693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 785/1570] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.444557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.450691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 786/1571] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:54.453349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.456132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec03d52-d65d-4df2-a0f1-0b0b057a88d0 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.462328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.463122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.463357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.463636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.463865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e5750-2af7-4a73-9864-04508e3fc8b1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.464327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.466527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.468647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.469284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.469568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.469845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.470561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:11:54.470876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.472482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.484708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 787/1572] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:11:54.485513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.497036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab397-aa8a-4660-8d84-e8edaf4215e8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.500164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 786/1573] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.501004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.504716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.505366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.505664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.505917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.506603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.509188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.538567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 788/1574] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.538567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.577222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669bbb-a4c4-4b78-bbad-70c08d139699 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.603831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb7a23-0f15-4fc1-97e0-be8c1942af04 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.608438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.609123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.609412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.609693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.610386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.611521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.612322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75699f0e-956b-49f2-9562-b9c6aa7f1bf0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.627480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.637531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.638206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.638494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.638773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.639471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.641429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.641700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 789/1575] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.643059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.643354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.643636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.661016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:54.662238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:54.667379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 787/1576] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.668319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.724923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 790/1577] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:54.729247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.734365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9c6f0-a4b9-4323-95a6-dc3ae84d0668 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.738773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac314f-2746-4d77-9a0e-799e045936e0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.739034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.739718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.740004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.740281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.741010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.743465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.744101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.745020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.745311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.745592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] PATH_INFO: `/v3/projects/edcf57a6279243308232f9fbc8c158e2/users/24f3f03c71d94f56aefcf930f55c352a/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.761706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcf57a6279243308232f9fbc8c158e2, user_id=24f3f03c71d94f56aefcf930f55c352a, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.764835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.771567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 788/1578] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.775899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.779401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 791/1579] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:54 2022] PUT /identity/v3/projects/edcf57a6279243308232f9fbc8c158e2/users/24f3f03c71d94f56aefcf930f55c352a/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:54.780252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.842622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13a90b-6a8f-4c47-af34-0564d74b39d4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.846852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261618-4b36-4c35-8041-ef450b2ff7b5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.847233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.847893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.848178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.848491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.849185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.851145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.851588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.852392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.852796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.853141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.853831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.855743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.873211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 792/1580] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.874272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.885929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37d68-7b39-4bfb-83c4-0938c185752f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.887736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 789/1581] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.888567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.924279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.947895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.948577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.948869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.949151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.949849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.951810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.954816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed77e-6576-44f9-ac82-0a87b6d32391 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.959660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:54.960020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.960972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.961264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.961716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.962767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.964701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:54.978055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 793/1582] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.978055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.987342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ca1-a3ae-43f0-afdb-c68e9bdac786 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:54.988771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 790/1583] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:54.992007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:54.992735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:54.993657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:54.993894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:54.994176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:54.994865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:54.996864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.021199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 794/1584] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:54 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:55.021199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.030920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc1ccf-9fd3-4082-808c-70a08b8b9edb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.035579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.036222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.036537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.036819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.037499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.039428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252961b-d9c9-492c-8722-6cd2927a836b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.045855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682b204-1e63-4bf0-baa9-e5c6ee385655 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.049592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.050244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.050536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.050814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.061102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.062283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 795/1585] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/users => generated 285 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:55.062524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.063530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.063823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.064105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.072399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 791/1586] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:55.073371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.083169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68b987-019c-4904-901a-f29e806c629e service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.087969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.088670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.088956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.089233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.089987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.092124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.098796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] MFA Rules not processed for user `24f3f03c71d94f56aefcf930f55c352a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:55.100350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:55.106954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 792/1587] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.108616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.122024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcceb38-899e-4205-96f9-7cbd87914f3b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.126331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.126991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.127283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.127562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.128258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.130232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.153612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 793/1588] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/users => generated 303 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:55.154491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.166924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2e71b-e77c-43bd-97fa-cb405edf6945 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.172007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.172788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.173186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.173540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.174337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.176007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.184515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 796/1589] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:55.184515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.200471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b155b-4bf4-4b1b-95d2-d50ea4e29076 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.205709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.206660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.206955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.207237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.207955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 794/1590] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.208070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.209580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.210814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.224705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad514a3c-5e0f-4e67-9c75-0c15f497652e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.228177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 797/1591] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.229220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.233311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.234258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.234679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.235082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.235938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.237709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.241522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b284-379c-4248-b2eb-34c19bf655bf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.250017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.250704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.252539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.252683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 795/1592] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.252781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.252880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.252957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.253911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.265928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376937a-d923-42d5-b995-e90606cc8552 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.268307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 798/1593] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.272101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.272511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.273223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.273784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.275056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.276265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.278194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.284040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea8814-a8a2-4227-ab79-24e2a2ca6278 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.294785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 796/1594] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.295007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.295177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.295495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.295495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.295495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.295495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.296687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541c57d-43e7-4072-b818-65c680f3792f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.306478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16502d5-66a3-4f69-91ed-b97f249b8b12 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.314036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 799/1595] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.314191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.314191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.314191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.314191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.314511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afa84-d5f9-4dc3-9cec-3d649bf8bfd8 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.314511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afa84-d5f9-4dc3-9cec-3d649bf8bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.314511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afa84-d5f9-4dc3-9cec-3d649bf8bfd8 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.314730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.314815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 800/1596] 10.222.0.41 () {54 vars in 961 bytes} [Fri May 27 15:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:11:55.315437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4afa84-d5f9-4dc3-9cec-3d649bf8bfd8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.315598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.327536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 797/1597] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.328588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.330310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4afa84-d5f9-4dc3-9cec-3d649bf8bfd8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.336054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.337202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.337791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.338181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.339050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.340390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0eec9-dee8-4475-9653-7677d8d5fa0c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.341524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.346915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.347614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.347906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.348313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.349358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.351155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.353992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 801/1598] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.355133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.362683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 798/1599] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.363629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.368773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59cdec-95fe-4f93-ba72-ee969f0c27c4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.375458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.376765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.377075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.377430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.379184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd7054-5c01-4661-86cc-359b1b55cfe9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.380297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.382448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.388540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.389932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.389932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.390146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] PATH_INFO: `/v3/projects/cc1a70bd2ce64fbcb738a2cfcfb09f41/users/ced10e76e5cc4443819157c821f82226/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.400648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 802/1600] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.401842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.407882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1a70bd2ce64fbcb738a2cfcfb09f41, user_id=ced10e76e5cc4443819157c821f82226, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.409092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:11:55.409092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:11:55.409092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:11:55.411185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a90eff-cc81-4c4c-9199-5559b3f875f7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.417593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5b045-8c15-4fd9-843e-575ad3dc08aa admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.423578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.424251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.424604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.424935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.425607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.431901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.432073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 799/1601] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/cc1a70bd2ce64fbcb738a2cfcfb09f41/users/ced10e76e5cc4443819157c821f82226/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.432073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.432073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.432073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.441245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 803/1602] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:55.441905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.451872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:55.452309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1503b2-d70b-44c0-a8bc-74587bf6f835 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.453066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:55.468044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:55.497602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:55.518095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.519007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.519286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.519561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] PATH_INFO: `/v3/projects/7f3d57e9e08741e8accf02c610c38068/users/c68f0e3afdc0445b9eaede54b7fbd079/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.536300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3d57e9e08741e8accf02c610c38068, user_id=c68f0e3afdc0445b9eaede54b7fbd079, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.538829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.553755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 800/1603] 10.222.0.41 () {58 vars in 1058 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:55.554970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.568994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ab49f-cc5a-47a6-8a6c-2b26739f534f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.575514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.576598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.576905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.577190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] PATH_INFO: `/v3/projects/ef917e1df4a64838aba088201ddb8552/users/bf2f8a02c0ee46029b08a31d7ee9eec4/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.591796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 804/1604] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/7f3d57e9e08741e8accf02c610c38068/users/c68f0e3afdc0445b9eaede54b7fbd079/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.591796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.606313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef917e1df4a64838aba088201ddb8552, user_id=bf2f8a02c0ee46029b08a31d7ee9eec4, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.609400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.628080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 801/1605] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/ef917e1df4a64838aba088201ddb8552/users/bf2f8a02c0ee46029b08a31d7ee9eec4/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.632585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.664003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef52b2-f11a-466c-9061-3091cfcfea77 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.707427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bd83c-0b4d-449c-b793-aa1392404a23 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.714046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.715085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.715455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.715724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] PATH_INFO: `/v3/projects/6712c1f221af4ed184db0fd21d094186/users/993ad0ad7aad4dfab555361cc85d788a/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.733928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6712c1f221af4ed184db0fd21d094186, user_id=993ad0ad7aad4dfab555361cc85d788a, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.734297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.735197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.735482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.735760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] PATH_INFO: `/v3/projects/72d5173b32b5480790babb3d178b94fb/users/097084a3e57540d18bae448da3b3ecc4/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.736720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa0eca-f83b-4355-9e12-ca317eacdc01 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.752230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d5173b32b5480790babb3d178b94fb, user_id=097084a3e57540d18bae448da3b3ecc4, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.752498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 802/1606] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/6712c1f221af4ed184db0fd21d094186/users/993ad0ad7aad4dfab555361cc85d788a/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.756516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.756516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.756516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.756845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.776818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 805/1607] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/72d5173b32b5480790babb3d178b94fb/users/097084a3e57540d18bae448da3b3ecc4/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.777642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.782161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:55.783382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:55.848052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:55.851716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 803/1608] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:55.851716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:55.862680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:55.867303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437574ad-0a54-4708-9c7a-ab668816ff7d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.875936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:55.877183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.877531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.877820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] PATH_INFO: `/v3/projects/267ba61d318543358dfc82753098ec7c/users/1b2425d0a41e44a38b64aa1a2526b276/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.905657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267ba61d318543358dfc82753098ec7c, user_id=1b2425d0a41e44a38b64aa1a2526b276, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:55.911089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d790a-37f4-461d-afe9-a5b20145b989 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:55.929424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 806/1609] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:55 2022] PUT /identity/v3/projects/267ba61d318543358dfc82753098ec7c/users/1b2425d0a41e44a38b64aa1a2526b276/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:55.930635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3316ecd-4e72-478f-832e-809e07148807 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:55.931153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3316ecd-4e72-478f-832e-809e07148807 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:55.931556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3316ecd-4e72-478f-832e-809e07148807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:55.964240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d3316ecd-4e72-478f-832e-809e07148807 None None] MFA Rules not processed for user `c68f0e3afdc0445b9eaede54b7fbd079`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:55.964428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c273e2-3546-4166-86c5-6b4124e625e3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:55.967859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d3316ecd-4e72-478f-832e-809e07148807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.054601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.056158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.056715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.057280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.065061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 807/1610] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:56.065061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87d8ae-ba46-49a9-8fe5-474647ad495f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.065061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87d8ae-ba46-49a9-8fe5-474647ad495f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.065061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87d8ae-ba46-49a9-8fe5-474647ad495f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.091218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cd87d8ae-ba46-49a9-8fe5-474647ad495f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:56.092785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cd87d8ae-ba46-49a9-8fe5-474647ad495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.137748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.140906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d1afe-db3d-4428-81cf-837ec421eec5 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.154347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 804/1611] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:56.156175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6598f-8dc9-4eec-a419-59ebc540e552 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.156870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6598f-8dc9-4eec-a419-59ebc540e552 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.157159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6598f-8dc9-4eec-a419-59ebc540e552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.179657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 808/1612] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:56.182184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.182611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.183085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.192051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1cb6598f-8dc9-4eec-a419-59ebc540e552 None None] MFA Rules not processed for user `993ad0ad7aad4dfab555361cc85d788a`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:56.193614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1cb6598f-8dc9-4eec-a419-59ebc540e552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.203796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:56.205243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.217299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 809/1613] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:56.218289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.248637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:56.291765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 805/1614] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:56.293423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.293782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.294131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.315260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:56.316679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.326400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 806/1615] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:56.327349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.349913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6038728-046b-40d4-8d5b-290c3f4601ce None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.352479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122662b2-18ba-4205-8c55-8336e9e46832 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.357643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.357826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.358314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.358495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.358603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.358785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.358895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.359075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.359894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.362268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.376542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.377604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.393086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 810/1616] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:56.394195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.395892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:56.411863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:56.418856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 807/1617] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:56.421876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.424599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd1a-f666-47ee-97f5-9ea71c1f077c service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.494928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.496060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.497235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.497610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.522616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84725f-5656-4c9d-a661-98441f7c4fc8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.529558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.530428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.530782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.531169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.604857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.607609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.622769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 811/1618] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:56.623691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.629619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.631556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.637894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681792b1-584c-49b9-ac0e-abb5f8cf6534 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.644126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.644405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.644725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.645006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.645708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.648121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaa601-617d-434c-b730-1c014b8009cc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.663379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 808/1619] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:56.665960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.676877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef149-33c1-4591-b1b5-87c69e0d5c12 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.684372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.686278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.686902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.687677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.702456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.704931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.720648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 809/1620] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:56.722283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.737172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 812/1621] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:56.737172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37314-649b-47e6-95fb-a8494fee2bac None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.737172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37314-649b-47e6-95fb-a8494fee2bac None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.737172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37314-649b-47e6-95fb-a8494fee2bac None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.740383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 813/1622] 10.222.0.41 () {54 vars in 912 bytes} [Fri May 27 15:11:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:11:56.742793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e37314-649b-47e6-95fb-a8494fee2bac None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:56.744669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e5537-aa36-426e-a952-d748edb3f68a service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.859677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:56.899134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e37314-649b-47e6-95fb-a8494fee2bac None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:56.902202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.902884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.909255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:56.910331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.910808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.911253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.912860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.916147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce1edc-5121-4bec-a0c2-2c4950c71a5f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.917113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.918268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.920054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:56.927630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:56.960561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 814/1623] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:56.960561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:56.960561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:56.960561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:56.989504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:56.990907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:57.000864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 810/1624] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:11:57.002762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.019253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:57.027133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c23c4-1cc1-4f06-a81a-3a62319cdd1a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.032493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.034914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.036158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.037568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.039301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.043205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.066512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 811/1625] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:57.067462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.083464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67388116-0adc-45a4-bd78-12d423a9e6cb admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.092419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.093365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.093670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.093987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.094732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.096422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.161120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 812/1626] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:57.162577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.165940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 815/1627] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:57.167259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.192860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed97a8-6203-47b4-b48f-3503502b0ed5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.204367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a255c5-e4c9-47a0-86c5-cdd85a483d49 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.208140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.209734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.210021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.210301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] PATH_INFO: `/v3/projects/0694d8f552f5474a8d760d9aa74ee277/users/85affb83b8084429a9c9b27ccd632f54/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.220140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.222822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.223174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.223462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] PATH_INFO: `/v3/projects/a21d285972be4f8288b8c78c6072eef7/users/c239430cf08f4606917678cccbf59431/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.231839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0694d8f552f5474a8d760d9aa74ee277, user_id=85affb83b8084429a9c9b27ccd632f54, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.235234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.243210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21d285972be4f8288b8c78c6072eef7, user_id=c239430cf08f4606917678cccbf59431, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.246755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.261386 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 813/1628] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:57 2022] PUT /identity/v3/projects/0694d8f552f5474a8d760d9aa74ee277/users/85affb83b8084429a9c9b27ccd632f54/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:57.262657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.278053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 816/1629] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:57 2022] PUT /identity/v3/projects/a21d285972be4f8288b8c78c6072eef7/users/c239430cf08f4606917678cccbf59431/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:57.278834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.386435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962c1f5-235f-4b97-9c1d-86e5cab02fb8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.400615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4db18-b0ea-4744-a829-f292ddddb892 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.416993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.418698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.419001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.419284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.420675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.421730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.446511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 817/1630] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:57.447815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.471103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2d84-437b-455f-99b5-59ec5b25c059 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.484894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.485579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.485846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.486122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.486854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.488605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.505667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 818/1631] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:11:57.505667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.507101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.509263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.510217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.510781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.515837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a854150-11ad-489b-bf65-0c03a8c9dc99 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.522253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.523259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.523547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.523867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] PATH_INFO: `/v3/projects/0694d8f552f5474a8d760d9aa74ee277/users/85affb83b8084429a9c9b27ccd632f54/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.526932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0694d8f552f5474a8d760d9aa74ee277, user_id=85affb83b8084429a9c9b27ccd632f54, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.530488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.565356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 819/1632] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:57 2022] PUT /identity/v3/projects/0694d8f552f5474a8d760d9aa74ee277/users/85affb83b8084429a9c9b27ccd632f54/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:57.567104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.597400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:57.636872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.638460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8da962-8a9b-4ab9-a212-d4256701dd01 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.656549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 814/1633] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:57.657666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46717788-ca25-46a8-8d15-4bee83e89fe6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:57.658162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.658162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.658456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.666661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:57.668650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.669158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.669562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] PATH_INFO: `/v3/projects/a21d285972be4f8288b8c78c6072eef7/users/c239430cf08f4606917678cccbf59431/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.672141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21d285972be4f8288b8c78c6072eef7, user_id=c239430cf08f4606917678cccbf59431, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:57.675423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793f396-4b67-456a-8bbd-107b5b1dbe38 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:57.678433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:57.699781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 820/1634] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:11:57 2022] PUT /identity/v3/projects/a21d285972be4f8288b8c78c6072eef7/users/c239430cf08f4606917678cccbf59431/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:57.712888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:57.713128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:57.713386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:57.714033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] MFA Rules not processed for user `85affb83b8084429a9c9b27ccd632f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:57.715268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:57.745935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] MFA Rules not processed for user `c239430cf08f4606917678cccbf59431`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:57.747394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:57.872520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 821/1635] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:11:57.882132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 815/1636] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:57.888114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:57.994741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac92536-4796-46a5-9de6-4159825ed526 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:58.000009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:58.001281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:58.001837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:58.001837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:58.016702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:58.018720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:58.052348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 822/1637] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:58.114057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:58.129807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4cb65-017c-4546-bae6-14b38ed53111 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:58.135949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:58.136929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:58.138139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:58.138433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:58.154848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:58.157400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c9b58-7b93-4862-b9d5-717555efb59c service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:58.189337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 816/1638] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:58 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:58.391538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:58.405109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bc8ee-3ad5-4b13-81fc-23cb2d48bf7b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:58.411342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:58.414127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:58.414127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:58.417212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:58.615052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:58.620633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b8790-e4c4-4d9c-bda0-d561eae3a1ac service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:58.651178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 823/1639] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:11:59.401569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550652cd-0c36-4774-aae7-9b532248af75 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.402576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550652cd-0c36-4774-aae7-9b532248af75 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.403685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550652cd-0c36-4774-aae7-9b532248af75 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.406033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 817/1640] 10.222.0.41 () {54 vars in 961 bytes} [Fri May 27 15:11:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:11:59.436259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d535-100b-4d9c-b345-3ef9d21ba93d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.436874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d535-100b-4d9c-b345-3ef9d21ba93d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.436874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d535-100b-4d9c-b345-3ef9d21ba93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.478270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8288d535-100b-4d9c-b345-3ef9d21ba93d None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:59.479520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8288d535-100b-4d9c-b345-3ef9d21ba93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:59.509926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:59.527816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550652cd-0c36-4774-aae7-9b532248af75 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:59.694917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 824/1641] 10.222.0.41 () {58 vars in 1058 bytes} [Fri May 27 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:59.701361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de79f-4610-4427-9b89-51364902987e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.701654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de79f-4610-4427-9b89-51364902987e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.701933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60de79f-4610-4427-9b89-51364902987e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.705772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 825/1642] 10.222.0.41 () {54 vars in 912 bytes} [Fri May 27 15:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:11:59.709968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2decc-7993-44a6-8400-30d35dac1417 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.710320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2decc-7993-44a6-8400-30d35dac1417 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.710607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2decc-7993-44a6-8400-30d35dac1417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.719496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550652cd-0c36-4774-aae7-9b532248af75 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:11:59.733263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:11:59.734511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.734792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.735072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] PATH_INFO: `/v3/users/c68f0e3afdc0445b9eaede54b7fbd079` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.738205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68f0e3afdc0445b9eaede54b7fbd079)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:11:59.739254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: WARNING py.warnings [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:11:59.739254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: warnings.warn(msg) May 27 15:11:59.739254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: May 27 15:11:59.740520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:11:59.747803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-26b2decc-7993-44a6-8400-30d35dac1417 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:59.749174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-26b2decc-7993-44a6-8400-30d35dac1417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:59.777393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 826/1643] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:11:59.778486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6dd1-cf82-4edd-9269-5b03717c79aa None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.778776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6dd1-cf82-4edd-9269-5b03717c79aa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.779055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6dd1-cf82-4edd-9269-5b03717c79aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:11:59.845051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-929a6dd1-cf82-4edd-9269-5b03717c79aa None None] MFA Rules not processed for user `1b2425d0a41e44a38b64aa1a2526b276`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:11:59.846191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-929a6dd1-cf82-4edd-9269-5b03717c79aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:59.851142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68f0e3afdc0445b9eaede54b7fbd079'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:11:59.877661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68f0e3afdc0445b9eaede54b7fbd079'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:11:59.878731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68f0e3afdc0445b9eaede54b7fbd079'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:11:59.881251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:59.911406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68f0e3afdc0445b9eaede54b7fbd079'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:11:59.922896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68f0e3afdc0445b9eaede54b7fbd079'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:11:59.931707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 818/1644] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:11:59 2022] DELETE /identity/v3/users/c68f0e3afdc0445b9eaede54b7fbd079 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:59.932765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:11:59.935682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:11:59.978414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 827/1645] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:11:59.980673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:11:59.981008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:11:59.981351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.007324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19fc12-4337-4943-aef7-dd5c4ba525e5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.013063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.013935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.014375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.014757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.016838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.021274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] MFA Rules not processed for user `bf2f8a02c0ee46029b08a31d7ee9eec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:00.022489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.163353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:00.166995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5926505-7dbe-41c7-a564-5f202c7b1616 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:00.185107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 819/1646] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:11:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:00.187213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.187515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.188185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.189944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 828/1647] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:00.198438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:00.205608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.231563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:00.235814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.246749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:00.306679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907042f-e6eb-44ed-9c30-db9b178e4fd7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.312561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.313418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.313817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.314401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.315050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:00.317570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e03f3b-55f6-4c51-9f3b-112778bb1cff admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:00.401275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 829/1648] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:00 2022] POST /identity/v3/projects => generated 403 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:00.405793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.406097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.407861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.421671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 820/1649] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:00.422659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:00.450486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a11ee-be16-42eb-b4ea-b45d3f6e8f76 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.477813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] MFA Rules not processed for user `bf2f8a02c0ee46029b08a31d7ee9eec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:00.480303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.481621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:00.607586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.609730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.610800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.612802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.613643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:00.618851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:00.660363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 830/1650] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:00.661387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:00.665553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 821/1651] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:00 2022] POST /identity/v3/users => generated 308 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:00.666484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:00.669777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.674229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:00.841828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a7d8c-2988-4fd0-baa2-d45c591cdf55 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.861989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.863280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.863773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.864791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.883172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:00.887109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:00.914180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 831/1652] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:00.917763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:00.966815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654b305-26e6-48b2-be01-fcf7601d8689 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.969262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c553a-062a-47a2-9f7c-d61285f759ab service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:00.973700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.974065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.974202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.974513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.979575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:00.979952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:00.983482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:00.983482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:00.984375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:00.990483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:00.999737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:01.007455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:01.010612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 832/1653] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:01.015869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:01.060781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 822/1654] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:01.063882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05787c61-9140-4aef-9f3a-4207e17f50d4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:01.080482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:01.081714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:01.082010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:01.082921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] PATH_INFO: `/v3/projects/f7069066f7b541519f7f4ed54378adcf/users/853955c46f604107bb55b587a94308ed/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:01.113405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7069066f7b541519f7f4ed54378adcf, user_id=853955c46f604107bb55b587a94308ed, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:01.119228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:01.151420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 833/1655] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:01 2022] PUT /identity/v3/projects/f7069066f7b541519f7f4ed54378adcf/users/853955c46f604107bb55b587a94308ed/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:01.151705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:01.174961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86e766-6107-4dde-9657-f07fe0d5c8b4 service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:01.356217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:01.357285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:01.359536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:01.360552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:01.377924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:01.382678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:01.414136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 823/1656] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:01.419456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:01.430719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2197b2-a8be-4348-a63f-9f12ddeb6e9e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:01.454561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:01.455461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:01.455750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:01.456030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] PATH_INFO: `/v3/projects/f7069066f7b541519f7f4ed54378adcf/users/853955c46f604107bb55b587a94308ed/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:01.464510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7069066f7b541519f7f4ed54378adcf, user_id=853955c46f604107bb55b587a94308ed, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:01.467470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d96f3-e574-452d-a9c2-fa7324de41a3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:01.500319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 834/1657] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:01 2022] PUT /identity/v3/projects/f7069066f7b541519f7f4ed54378adcf/users/853955c46f604107bb55b587a94308ed/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:01.833134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:01.837729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:01.866453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e935ac-48fa-485e-9921-e39509274b0b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:01.929713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:02.015614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:02.018599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.019363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.022705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] PATH_INFO: `/v3/users/24f3f03c71d94f56aefcf930f55c352a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.023730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f3f03c71d94f56aefcf930f55c352a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:02.032092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:02.138330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f3f03c71d94f56aefcf930f55c352a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:02.155947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f3f03c71d94f56aefcf930f55c352a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:02.157690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f3f03c71d94f56aefcf930f55c352a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:02.197898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f3f03c71d94f56aefcf930f55c352a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:02.210569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a877570-59c2-444d-ad10-f7c9eb49cce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f3f03c71d94f56aefcf930f55c352a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:02.223732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 824/1658] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:01 2022] DELETE /identity/v3/users/24f3f03c71d94f56aefcf930f55c352a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:02.228946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.229260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.229544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.272393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:02.276907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:02.287890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:02.424976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec631f-c691-4397-82d1-247a8ea660ab None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.425288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec631f-c691-4397-82d1-247a8ea660ab None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.425566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec631f-c691-4397-82d1-247a8ea660ab None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.426952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 825/1659] 10.222.0.41 () {54 vars in 912 bytes} [Fri May 27 15:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:12:02.434317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.434939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.435362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.468548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 835/1660] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:02.557681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:02.562403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:02.607935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:02.609417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:02.702510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 826/1661] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:02.734700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:02.736370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:02.737637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03d762-7e51-4330-891c-d26bd5e579d2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:02.753495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417447c-4a18-4cf2-96da-627d9f43729e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:02.761173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:02.761781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.762070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.762346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.781714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:02.783393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:02.783530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:02.783677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:02.783956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:02.785227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:02.813066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:02.815038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:02.820134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 836/1662] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:02.841931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 827/1663] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:03.724204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:03.728996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:03.779006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104462d3-18b5-41c1-aba0-4090672d57ec service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:03.808305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:03.811152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:03.811618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:03.813784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] PATH_INFO: `/v3/projects/7f3d57e9e08741e8accf02c610c38068` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:03.817359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3d57e9e08741e8accf02c610c38068)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:03.818748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: WARNING py.warnings [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:12:03.818748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: warnings.warn(msg) May 27 15:12:03.818748 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: May 27 15:12:03.824913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:03.953144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:03.956276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.020135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae50b43-df7c-430c-bf4e-97e9009d2fec service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:04.026557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:04.027452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.027736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.028015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] PATH_INFO: `/v3/users/993ad0ad7aad4dfab555361cc85d788a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.048220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993ad0ad7aad4dfab555361cc85d788a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:04.050565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:04.089497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3d57e9e08741e8accf02c610c38068', 'request_id': 'req-b302bea7-ba15-43c8-8abc-1f46420890bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.128177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:04.150099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3d57e9e08741e8accf02c610c38068', 'request_id': 'req-b302bea7-ba15-43c8-8abc-1f46420890bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.166551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3d57e9e08741e8accf02c610c38068', 'request_id': 'req-b302bea7-ba15-43c8-8abc-1f46420890bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.186844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] The token cache is being invalidate because project 7f3d57e9e08741e8accf02c610c38068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:04.187168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.187996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993ad0ad7aad4dfab555361cc85d788a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.188372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b302bea7-ba15-43c8-8abc-1f46420890bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.197835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993ad0ad7aad4dfab555361cc85d788a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.198944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993ad0ad7aad4dfab555361cc85d788a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.201214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 837/1664] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:03 2022] DELETE /identity/v3/projects/7f3d57e9e08741e8accf02c610c38068 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:04.202706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.203005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.203283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.214131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 838/1665] 10.222.0.41 () {54 vars in 960 bytes} [Fri May 27 15:12:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:04.214131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:04.219228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.234346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993ad0ad7aad4dfab555361cc85d788a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.260490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27b8b9c7-8029-4d8d-b3e5-53855a4f4632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993ad0ad7aad4dfab555361cc85d788a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.280489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 828/1666] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:03 2022] DELETE /identity/v3/users/993ad0ad7aad4dfab555361cc85d788a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:04.284080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4c50-2c4c-4a14-a665-f4ec245fd5b9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.289064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4c50-2c4c-4a14-a665-f4ec245fd5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.290496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4c50-2c4c-4a14-a665-f4ec245fd5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.294803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:04.322503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-32ae4c50-2c4c-4a14-a665-f4ec245fd5b9 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:04.324415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-32ae4c50-2c4c-4a14-a665-f4ec245fd5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.342638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dd001-878e-4a80-a879-0d80add38b7a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:04.464290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 829/1667] 10.222.0.41 () {58 vars in 1057 bytes} [Fri May 27 15:12:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:04.467311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.468079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.468814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.485731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:04.487724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.487920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.488200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] PATH_INFO: `/v3/users/c239430cf08f4606917678cccbf59431` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.493702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c239430cf08f4606917678cccbf59431)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:04.496592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:04.507264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:04.510337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.529052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 830/1668] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:04.529052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:04.544295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f59dac-329c-41da-9c20-02653873ca5b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:04.565909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:04.568010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.568421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.569033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.579280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.609121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:04.609548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c239430cf08f4606917678cccbf59431'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.610772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:04.621641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c239430cf08f4606917678cccbf59431'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.622842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c239430cf08f4606917678cccbf59431'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.633672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 831/1669] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:04.654810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:04.656692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:04.675392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c239430cf08f4606917678cccbf59431'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.676754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c239430cf08f4606917678cccbf59431'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:04.729079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 839/1670] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:04 2022] DELETE /identity/v3/users/c239430cf08f4606917678cccbf59431 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:04.807597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233854db-359c-49a3-a355-6d4e01a873f2 service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:04.853796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:04.895374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:04.896203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:04.896683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:04.897061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:04.899185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:05.047796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:05.048981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:05.075069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 832/1671] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:06.950359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:06.951883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:06.982934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9c2c5-1bcb-4827-b149-9f46caaac115 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:06.998285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64457d20-80af-4226-a442-420c346166aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:06.999199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64457d20-80af-4226-a442-420c346166aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:06.999637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64457d20-80af-4226-a442-420c346166aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:06.999927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64457d20-80af-4226-a442-420c346166aa admin admin] PATH_INFO: `/v3/projects/a21d285972be4f8288b8c78c6072eef7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.004845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64457d20-80af-4226-a442-420c346166aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21d285972be4f8288b8c78c6072eef7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:07.007018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64457d20-80af-4226-a442-420c346166aa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:07.034629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:07.036324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.077965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242828f2-7caa-4dce-8060-2d38b67d7549 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:07.105326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:07.108580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.108941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.109237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] PATH_INFO: `/v3/projects/6712c1f221af4ed184db0fd21d094186` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.117413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6712c1f221af4ed184db0fd21d094186)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:07.119418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:07.178842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21d285972be4f8288b8c78c6072eef7', 'request_id': 'req-64457d20-80af-4226-a442-420c346166aa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.207582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21d285972be4f8288b8c78c6072eef7', 'request_id': 'req-64457d20-80af-4226-a442-420c346166aa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.215156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:07.232262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21d285972be4f8288b8c78c6072eef7', 'request_id': 'req-64457d20-80af-4226-a442-420c346166aa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.286644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] The token cache is being invalidate because project a21d285972be4f8288b8c78c6072eef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:07.289142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.291405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-64457d20-80af-4226-a442-420c346166aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.298733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 840/1672] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:06 2022] DELETE /identity/v3/projects/a21d285972be4f8288b8c78c6072eef7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:07.302456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aa23f-6d55-4984-bf44-fb4d1efed47f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.302756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aa23f-6d55-4984-bf44-fb4d1efed47f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.303041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aa23f-6d55-4984-bf44-fb4d1efed47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.348474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-906aa23f-6d55-4984-bf44-fb4d1efed47f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:07.349950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-906aa23f-6d55-4984-bf44-fb4d1efed47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.350885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6712c1f221af4ed184db0fd21d094186', 'request_id': 'req-d974448d-eef2-4098-8b93-73ecc79a0192'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.377320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6712c1f221af4ed184db0fd21d094186', 'request_id': 'req-d974448d-eef2-4098-8b93-73ecc79a0192'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.392165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6712c1f221af4ed184db0fd21d094186', 'request_id': 'req-d974448d-eef2-4098-8b93-73ecc79a0192'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.425587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] The token cache is being invalidate because project 6712c1f221af4ed184db0fd21d094186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:07.425915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.431683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.436336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 833/1673] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:07 2022] DELETE /identity/v3/projects/6712c1f221af4ed184db0fd21d094186 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:07.437532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:07.438986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.488944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 841/1674] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:07.500475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.500936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.501409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.527974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:07.529266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.557701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 842/1675] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:07.568544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:07.601212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4900f7e-fa4c-4d56-ad18-c5d83d2e7812 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:07.629273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:07.629974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.630264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.630546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.631311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:07.631613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:07.633974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:07.650607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974448d-eef2-4098-8b93-73ecc79a0192 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:07.674963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:07.675876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.676157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.676434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] PATH_INFO: `/v3/users/85affb83b8084429a9c9b27ccd632f54` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.683225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 843/1676] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:07.683225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:07.683487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85affb83b8084429a9c9b27ccd632f54)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:07.683487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:07.697407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.704399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:07.747759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b244-65c7-4bab-ac0b-8c0e8512d71c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:07.778413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:07.779318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:07.781920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:07.782320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] PATH_INFO: `/v3/projects/edcf57a6279243308232f9fbc8c158e2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:07.786019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edcf57a6279243308232f9fbc8c158e2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:07.792955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:07.831629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85affb83b8084429a9c9b27ccd632f54'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.850066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85affb83b8084429a9c9b27ccd632f54'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.853182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85affb83b8084429a9c9b27ccd632f54'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.889636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85affb83b8084429a9c9b27ccd632f54'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.902106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85affb83b8084429a9c9b27ccd632f54'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.920437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 834/1677] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:07 2022] DELETE /identity/v3/users/85affb83b8084429a9c9b27ccd632f54 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:07.921663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:07.923674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:07.970699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edcf57a6279243308232f9fbc8c158e2', 'request_id': 'req-a142d913-80d6-4eb2-8081-cbcc83df5567'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.983460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edcf57a6279243308232f9fbc8c158e2', 'request_id': 'req-a142d913-80d6-4eb2-8081-cbcc83df5567'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:07.997215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edcf57a6279243308232f9fbc8c158e2', 'request_id': 'req-a142d913-80d6-4eb2-8081-cbcc83df5567'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:08.018820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] The token cache is being invalidate because project edcf57a6279243308232f9fbc8c158e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:08.019124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:08.024023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-a142d913-80d6-4eb2-8081-cbcc83df5567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:08.029421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 844/1678] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:07 2022] DELETE /identity/v3/projects/edcf57a6279243308232f9fbc8c158e2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:08.034231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb207-3193-446b-a65a-d652631f127c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.034756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb207-3193-446b-a65a-d652631f127c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.035190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfb207-3193-446b-a65a-d652631f127c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.083024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f8bfb207-3193-446b-a65a-d652631f127c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:08.088822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f8bfb207-3193-446b-a65a-d652631f127c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:08.099664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d6f39-4161-4c09-844f-95870f9bd10b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:08.175283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:08.176115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.177469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.177875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.179331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:08.184403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:08.207649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:08.231674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 845/1679] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:08.250824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b7eb8-e29e-44bf-8c72-dbd023835438 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.251913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b7eb8-e29e-44bf-8c72-dbd023835438 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.253356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b7eb8-e29e-44bf-8c72-dbd023835438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.282562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 835/1680] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:07 2022] POST /identity/v3/projects => generated 365 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:08.291524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:08.296995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:08.326641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b16b7eb8-e29e-44bf-8c72-dbd023835438 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:08.328073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b16b7eb8-e29e-44bf-8c72-dbd023835438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:08.372640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:08.599386 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef2e24-1d6e-418e-95c8-64304f8973a7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:08.617742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:08.619703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.621734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.624098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.626427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:08.627534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 846/1681] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:08.630027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.634143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.634604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.635156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:08.709076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 836/1682] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/users => generated 283 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:08.711556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:08.714349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:08.717393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:08.751908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bccbf0-af68-413b-99b5-cf87f2c36001 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:08.785643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:08.786247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.786538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.786819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.787574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:08.794282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:08.807229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:08.856729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 837/1683] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:08.857709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:08.912046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8361c-1327-4c0a-85be-1a4d82f1e70b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:08.933618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:08.935838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.936133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:08.936417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:08.953829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:08.956872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e71382f-2dcc-4e4e-b0e2-69e370ca10b2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:08.962213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 847/1684] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:08.966277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:08.998026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 838/1685] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:08.998696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd2e4a-1850-4ba5-97d3-5211295633ce None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:08.999375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:08.999671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.000132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.004905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.007655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.008786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.009187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.011497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.020775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.099136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] MFA Rules not processed for user `853955c46f604107bb55b587a94308ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:09.099342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 848/1686] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:09.100546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:09.101350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.294882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 839/1687] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:09.295861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.338946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cd266-2211-4348-9f3c-1e455b7c904b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.348581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.350341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.351019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.352219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.353281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.359227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.379153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 849/1688] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:09.385515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.387606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:09.400478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:09.435668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50234889-4c48-42b8-aeb6-6c332ad2206d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.442800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.444280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.445004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.446040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.447022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.450009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.472090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:09.482760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 840/1689] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:09 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:09.483983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.496206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61d249-bb78-479c-8b2c-6128104a8b17 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.502484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.503915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.504395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.505196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] PATH_INFO: `/v3/projects/4a91531b73624e06a99d8a425a9a5b83/users/e8ad5f3c7b8b48c0b59bf52d1d6e2599/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.527232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a91531b73624e06a99d8a425a9a5b83, user_id=e8ad5f3c7b8b48c0b59bf52d1d6e2599, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.531493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.551784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 841/1690] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:09 2022] PUT /identity/v3/projects/4a91531b73624e06a99d8a425a9a5b83/users/e8ad5f3c7b8b48c0b59bf52d1d6e2599/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:09.553903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.563753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089425-043c-40b8-95de-5272757f87db admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.698838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6f60-0c3f-4d35-b760-d9fffe5dd227 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.708051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.708281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.708527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.708807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.712250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.712250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.713148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.713545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.714295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.717073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06793c-7764-42c3-8b1f-7e8d4241187c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.744107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 842/1691] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:09.747040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.747338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.747623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.813704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] MFA Rules not processed for user `e8ad5f3c7b8b48c0b59bf52d1d6e2599`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:09.818899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:09.857170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.859570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:09.890850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 850/1692] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:12:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:12:09.893649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:09.898776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:09.939793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118302-c1f8-4378-bf6c-4915baf5b7af service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:09.959659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:09.960597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:09.960882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:09.961293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] PATH_INFO: `/v3/projects/0694d8f552f5474a8d760d9aa74ee277` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:09.971228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0694d8f552f5474a8d760d9aa74ee277)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:09.973299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:10.087743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 843/1693] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:10.088770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:10.126271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9741-4790-44db-8450-90964cb2e909 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:10.142616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:10.144315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:10.146434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:10.146667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:10.148382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:10.151408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:10.170320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 844/1694] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:10.178803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:10.193780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4124ae-0e25-467f-997a-ff5b0a5e19dc admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:10.265622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:10.273201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0694d8f552f5474a8d760d9aa74ee277', 'request_id': 'req-18f9c89d-33f3-45a6-8f46-1492f79aef2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:10.308613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0694d8f552f5474a8d760d9aa74ee277', 'request_id': 'req-18f9c89d-33f3-45a6-8f46-1492f79aef2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:10.317038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:10.335473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:10.336577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:10.337409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:10.337756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:10.338695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:10.342216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22324817-6dfb-471f-a2d1-81c9f069132d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:10.345496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0694d8f552f5474a8d760d9aa74ee277', 'request_id': 'req-18f9c89d-33f3-45a6-8f46-1492f79aef2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:10.402431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] The token cache is being invalidate because project 0694d8f552f5474a8d760d9aa74ee277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:10.402730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:10.412814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:10.425943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 851/1695] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:09 2022] DELETE /identity/v3/projects/0694d8f552f5474a8d760d9aa74ee277 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:10.439979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:10.441592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:10.442960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 845/1696] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:10 2022] POST /identity/v3/projects => generated 365 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:10.448935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:10.449227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:10.451556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:10.456730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9c89d-33f3-45a6-8f46-1492f79aef2a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:10.528427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:10.533553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:10.636288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:10.643355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:10.644961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:10.645192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] PATH_INFO: `/v3/projects/b041a9f731074ff6bc97593c9dab81db/users/6bb159f8906243ee9a6ec6c7c3988a99/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:10.749289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b041a9f731074ff6bc97593c9dab81db, user_id=6bb159f8906243ee9a6ec6c7c3988a99, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:10.763147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a140c-9580-43b4-a60a-469cb5ca9232 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:10.803687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 846/1697] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:10.814530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:10.835252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:10.838710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 852/1698] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:10 2022] PUT /identity/v3/projects/b041a9f731074ff6bc97593c9dab81db/users/6bb159f8906243ee9a6ec6c7c3988a99/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:10.853589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537968e-55e5-4305-8628-baeed214ff84 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:10.853894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537968e-55e5-4305-8628-baeed214ff84 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:10.854205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537968e-55e5-4305-8628-baeed214ff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:10.887012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e9b037-55f3-4476-ac65-a1b2337d2e46 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:10.888077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5537968e-55e5-4305-8628-baeed214ff84 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:10.898970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5537968e-55e5-4305-8628-baeed214ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:11.071597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.075900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.077524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.077972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.081768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.092419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.115653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 853/1699] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:11.115653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.115653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.115653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.168385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 847/1700] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:10 2022] POST /identity/v3/users => generated 289 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:11.169416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.196692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] MFA Rules not processed for user `6bb159f8906243ee9a6ec6c7c3988a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:11.211523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:11.218949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea6261-1d5b-4da5-92da-96dbf41d66cb admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.234374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.239299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.239597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.242770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.246973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:11.247466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.253859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.312858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 848/1701] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:11.313755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.356981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17186a03-f342-4ff8-90af-9e9ec4c70dfe admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.362384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:11.363698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.364628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.366471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.366842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.368560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.372125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.392559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 849/1702] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:11.393875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.413878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f109cb-a6f3-4f6f-b433-6862c08d880d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.421362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.422554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.423007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.423359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.424556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.426555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.435005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 854/1703] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:11.436074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.469610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596965aa-f4ae-4977-b062-7900fa66ce1f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.487131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.488054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.488345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.488655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] PATH_INFO: `/v3/projects/e5fd3a395a754024babe5389c65e54b7/users/c768cb426e264ef9bc382ada1c966444/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.496419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 850/1704] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:11 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:11.497532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.532971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fd3a395a754024babe5389c65e54b7, user_id=c768cb426e264ef9bc382ada1c966444, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.536191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.547749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:11.565567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 855/1705] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:11 2022] PUT /identity/v3/projects/e5fd3a395a754024babe5389c65e54b7/users/c768cb426e264ef9bc382ada1c966444/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:11.568266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.668804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a5394-f6a5-46bf-8f65-6f107c19f250 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.718561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329df10-0b26-47bd-8743-f19626f1fc0c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.727523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.728226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.728619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.728899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.729872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.757901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.842698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 856/1706] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:11 2022] POST /identity/v3/users => generated 294 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:11.843366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.857524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.858243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.858587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.858931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.859740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.865377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.879216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033c03c-3508-4558-8921-e86c05464932 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:11.887324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:11.888204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:11.888680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:11.889108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:11.890105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:11.892303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:11.908640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 857/1707] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:11.911235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.927899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 851/1708] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:11 2022] POST /identity/v3/projects => generated 371 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:11.932590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:11.938338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743b0ed-ac53-487f-bbfb-9a3e1d6f4fea admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.118722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.119948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.120388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.120855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.121756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.123627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:12.152180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 858/1709] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:12.153632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:12.172636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53cba-f333-4d38-9de3-36fc9e1cf975 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.190727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.191741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.192232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.192721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.193881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.201319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:12.282928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 859/1710] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:12 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:12.282928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:12.297385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e3655-16bd-4de0-9f91-ae7e24bd01b8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.308469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.309647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.310779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.311285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.312759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.315958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39bf0a7-5de3-4e82-9e7d-4a7e57b535b2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:12.330838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 860/1711] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:12.332972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.333440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.334465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.363831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:12.401728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] MFA Rules not processed for user `097084a3e57540d18bae448da3b3ecc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:12.402993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:12.434152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb79f-b370-4a25-823d-370a1597b909 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.443107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.444359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.444694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.445171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] PATH_INFO: `/v3/projects/e5fd3a395a754024babe5389c65e54b7/users/c768cb426e264ef9bc382ada1c966444/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.448689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fd3a395a754024babe5389c65e54b7, user_id=c768cb426e264ef9bc382ada1c966444, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.451828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:12.473650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 852/1712] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:11 2022] PUT /identity/v3/projects/e5fd3a395a754024babe5389c65e54b7/users/c768cb426e264ef9bc382ada1c966444/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:12.487000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:12.555727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 861/1713] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:12.556818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:12.603143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba55e1-a65f-49fe-a053-17f5dc11dbcc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.697659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.698583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.699473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.699473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] PATH_INFO: `/v3/projects/e932721cf3cc42b8bff32e55e6d485a6/users/45cba2e0d2d7469694dc82b3a26cf53c/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.729820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:12.737718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e932721cf3cc42b8bff32e55e6d485a6, user_id=45cba2e0d2d7469694dc82b3a26cf53c, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.741610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae9a7d-1ff8-4b36-8b26-b3ef09c83ffc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:12.775421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 862/1714] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:12 2022] PUT /identity/v3/projects/e932721cf3cc42b8bff32e55e6d485a6/users/45cba2e0d2d7469694dc82b3a26cf53c/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:12.780025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128411-c75f-436a-bdcf-dc11a6a0ab96 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.781972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128411-c75f-436a-bdcf-dc11a6a0ab96 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.782280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128411-c75f-436a-bdcf-dc11a6a0ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.796116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d03b1-7b45-48a2-a52b-4672a3cfe87a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:12.875598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4c128411-c75f-436a-bdcf-dc11a6a0ab96 None None] MFA Rules not processed for user `c768cb426e264ef9bc382ada1c966444`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:12.881239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4c128411-c75f-436a-bdcf-dc11a6a0ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:12.954761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:12.955459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:12.955748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:12.956029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:12.962384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:12.967367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:13.002366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 853/1715] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:13.003715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:13.005741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:13.125736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 863/1716] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:13.129701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:13.130002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:13.130360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:13.175534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:13.201765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] MFA Rules not processed for user `45cba2e0d2d7469694dc82b3a26cf53c`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:13.205491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:13.216431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858015f-1cd1-4edf-b87b-1c5b6750f1ad admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:13.234644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:13.235420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:13.235726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:13.236026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:13.400407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:13.402248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:13.410950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 864/1717] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:13.413331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:13.415825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 854/1718] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:13.417130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:13.418681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:13.442529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7392754-79eb-4b0d-adfa-c9d2cf678e65 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:13.462125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:13.463454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:13.463786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:13.464108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] PATH_INFO: `/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15/users/46ed8fc893af4958b52bf588647c7a67/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:13.476222 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:13.502936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6a8c05361440c6a24b12d3d1d08c15, user_id=46ed8fc893af4958b52bf588647c7a67, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:13.505852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:13.536572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 865/1719] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:13 2022] PUT /identity/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15/users/46ed8fc893af4958b52bf588647c7a67/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:13.537328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:13.567549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e89a38-c422-4344-9912-6b36d6bffda6 service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:13.728042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:13.730664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:13.731032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:13.731319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:13.881579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:13.885117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dc4452-8276-4fca-91bc-8068b051ac5b service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:13.937481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 855/1720] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:13.957725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2e67-c7f9-4a90-9d41-a789b99b3516 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:13.958018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2e67-c7f9-4a90-9d41-a789b99b3516 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:13.958309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd2e67-c7f9-4a90-9d41-a789b99b3516 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:13.959812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 856/1721] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:12:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:13.960619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd2e67-c7f9-4a90-9d41-a789b99b3516 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.150566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd2e67-c7f9-4a90-9d41-a789b99b3516 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.168205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.168950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.169280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.169600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.170423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:14.172162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:14.183524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069eade-ede7-4316-8e5e-14a816a16157 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.193042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 857/1722] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:14.198180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.199521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.200639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.215723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.217059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.219533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:14.225267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:14.245459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dc72d-62ef-48b3-afe2-bb727e9372f3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.269696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.270611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.270893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.271171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] PATH_INFO: `/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15/users/46ed8fc893af4958b52bf588647c7a67/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.275430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6a8c05361440c6a24b12d3d1d08c15, user_id=46ed8fc893af4958b52bf588647c7a67, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:14.286617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c17c7-ea63-48ae-8996-b2145fa32154 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:14.312055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 858/1723] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:14 2022] PUT /identity/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15/users/46ed8fc893af4958b52bf588647c7a67/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:14.324145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.324435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.324807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.357073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 866/1724] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:13 2022] POST /identity/v3/projects => generated 383 bytes in 820 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:14.370733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.377890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:14.388861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] MFA Rules not processed for user `46ed8fc893af4958b52bf588647c7a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:14.390181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:14.563945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b85c3-8fa2-4ee7-ad21-b810b860d99c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.577083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.577951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.578332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.578757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.579736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:14.587505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa5a9f-0533-4bd2-af3e-f2a37753ab99 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:14.596120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:14.631588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 867/1725] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:14 2022] POST /identity/v3/users => generated 298 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:14.636145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.638540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.638944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.645980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 859/1726] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:14.646894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.692163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25d89c-c0cb-4aab-b6cd-83364b101325 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.711932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.712775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.713143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.713421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.714245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:14.716372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:14.719143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:14.725743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:14.771655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 860/1727] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:14.772989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.779043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:14.966355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 868/1728] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:14.969749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:14.971948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078700b-e043-4b5f-9ae8-6ddcdcd99e0f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:14.989354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:14.990116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:14.990424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:14.990702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:14.994704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387b136-5f31-4b24-a1d4-b989088429f9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.003262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.005650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.006175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.009750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] PATH_INFO: `/v3/projects/ae474d6b6ed94af987351e90515ba87c/users/87a4e5257aa8443fb564a8427e672196/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.019056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.020998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.045783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae474d6b6ed94af987351e90515ba87c, user_id=87a4e5257aa8443fb564a8427e672196, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.059960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 861/1729] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:15.063693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:15.067906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.121690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c020-abbf-4a22-9dca-9e5038a4eb8f service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.130904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 869/1730] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:14 2022] PUT /identity/v3/projects/ae474d6b6ed94af987351e90515ba87c/users/87a4e5257aa8443fb564a8427e672196/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:15.134397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:15.141834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:15.365189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.366091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.366372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.366676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] PATH_INFO: `/v3/users/e8ad5f3c7b8b48c0b59bf52d1d6e2599` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.370176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ad5f3c7b8b48c0b59bf52d1d6e2599)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.371988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.404315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df4487-326e-48fb-adcb-60c52043d3b0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.420659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.421343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.421639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.421924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.446659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.448340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.469711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 870/1731] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:15.469880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:15.489460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:15.489661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a32-1801-4467-b6ed-0eee104a7c21 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.501244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.502187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.502351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.502504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.503246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.509337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.532740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 871/1732] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:15.537426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:15.574623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ad5f3c7b8b48c0b59bf52d1d6e2599'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:15.603310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ad5f3c7b8b48c0b59bf52d1d6e2599'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:15.608485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ad5f3c7b8b48c0b59bf52d1d6e2599'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:15.656315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ad5f3c7b8b48c0b59bf52d1d6e2599'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:15.659300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:15.673401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ad5f3c7b8b48c0b59bf52d1d6e2599'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:15.685916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a155f2-4382-4314-877f-44ffa1fa1bbd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.689977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 862/1733] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:15 2022] DELETE /identity/v3/users/e8ad5f3c7b8b48c0b59bf52d1d6e2599 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:15.694106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:15.699882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:15.865739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd35d77-f75f-4ded-83c5-e64f2acfad7d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:15.866098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.866822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.870641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.871062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.873481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:15.879621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:15.880681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.881084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.881431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] PATH_INFO: `/v3/projects/ae474d6b6ed94af987351e90515ba87c/users/87a4e5257aa8443fb564a8427e672196/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.885548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae474d6b6ed94af987351e90515ba87c, user_id=87a4e5257aa8443fb564a8427e672196, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:15.896261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1c972-0873-4ea7-98d4-1ae7dcfdf44c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:15.936705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 863/1734] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:15 2022] PUT /identity/v3/projects/ae474d6b6ed94af987351e90515ba87c/users/87a4e5257aa8443fb564a8427e672196/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:15.941717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6c9b-6357-494f-b7b0-3d75508aed0c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:15.942020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6c9b-6357-494f-b7b0-3d75508aed0c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:15.942301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f6c9b-6357-494f-b7b0-3d75508aed0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:15.991250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c97f6c9b-6357-494f-b7b0-3d75508aed0c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:15.996391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c97f6c9b-6357-494f-b7b0-3d75508aed0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.039560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:16.047284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785f9da-88cd-422c-a0d2-b96d65ea8066 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:16.102483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 872/1735] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:16.110355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:16.113899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:16.114758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:16.180871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 864/1736] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:16.186790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8a9b4-08be-402d-8c95-5f1ebee69ad6 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:16.189352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8a9b4-08be-402d-8c95-5f1ebee69ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:16.189662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8a9b4-08be-402d-8c95-5f1ebee69ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:16.226042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] MFA Rules not processed for user `d25dc63e8a4c4af8af2fe29b7819212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:16.228488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.288569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e2f8a9b4-08be-402d-8c95-5f1ebee69ad6 None None] MFA Rules not processed for user `87a4e5257aa8443fb564a8427e672196`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:16.292681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e2f8a9b4-08be-402d-8c95-5f1ebee69ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.480196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 873/1737] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:16.480196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:16.480196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.532993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:16.643635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 865/1738] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:16.646373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:16.646663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:16.647018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:16.705429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] MFA Rules not processed for user `d25dc63e8a4c4af8af2fe29b7819212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:16.706732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.718233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:16.739696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 866/1739] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:16.752374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:16.761429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.784039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6defe84-b225-42db-97b0-dbce31965b1b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:16.794960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:16.798045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:16.798870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:16.799158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:16.839914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:16.843752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:16.871890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 874/1740] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:16.874897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:16.884098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:16.969202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feab99-2c43-45ac-b777-23588eac5c2f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:16.980601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:16.982018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:16.982304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:16.982584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.020651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:17.035909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53e132-2852-4e3a-97f6-87773841c3be service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:17.082908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 867/1741] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:17.088773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.089084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.089338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.185881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:17.191464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:17.199134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827542c-4c66-4edb-8396-e1ec72223d85 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:17.223564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:17.223945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 868/1742] 10.222.0.40 () {58 vars in 1015 bytes} [Fri May 27 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:17.225224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:17.229015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.231695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.232146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.259758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:17.262104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384192d5-4a8a-46dd-a6b9-8de6704a70e3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:17.266917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db429-0e73-4ab3-baa1-d336df4d510b service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:17.282561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:17.285521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.291234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.292198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] PATH_INFO: `/v3/users/45cba2e0d2d7469694dc82b3a26cf53c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.297971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45cba2e0d2d7469694dc82b3a26cf53c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:17.309784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:17.330625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 875/1743] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:17.419865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.420764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.421408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.455037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45cba2e0d2d7469694dc82b3a26cf53c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:17.464611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45cba2e0d2d7469694dc82b3a26cf53c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:17.466916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45cba2e0d2d7469694dc82b3a26cf53c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:17.485689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45cba2e0d2d7469694dc82b3a26cf53c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:17.496759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45cba2e0d2d7469694dc82b3a26cf53c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:17.509263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 869/1744] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:17 2022] DELETE /identity/v3/users/45cba2e0d2d7469694dc82b3a26cf53c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:17.512349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:17.512349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:17.516710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] MFA Rules not processed for user `097084a3e57540d18bae448da3b3ecc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:17.521631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:17.566971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:17.611311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467446e1-9741-47f1-a2af-add323e62c89 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:17.648301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:17.653177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.654254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.656147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.665503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:17.698275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:17.700434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c7048-cbfd-4b2d-9c17-d2668c81b9b4 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:17.713976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 870/1745] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:17.717871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7172a5-0abf-466f-aba0-7f232e25fc6b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:17.718345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7172a5-0abf-466f-aba0-7f232e25fc6b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:17.718820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7172a5-0abf-466f-aba0-7f232e25fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:17.724509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 876/1746] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:17.774225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-af7172a5-0abf-466f-aba0-7f232e25fc6b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:17.775790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-af7172a5-0abf-466f-aba0-7f232e25fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:17.988495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 871/1747] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:18.213278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:18.217135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:18.282470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687ff1d-a16f-4a38-a0ce-264879ab5f89 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:18.294950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:18.296710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:18.298685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:18.299216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:18.333060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:18.339289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0028aa0-72fa-4c7d-abee-e7edefa7522c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:18.389966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 877/1748] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:18.515465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2120eed-4f44-49d5-a893-cf1b275cba83 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:18.515877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2120eed-4f44-49d5-a893-cf1b275cba83 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:18.516324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2120eed-4f44-49d5-a893-cf1b275cba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:18.653556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a2120eed-4f44-49d5-a893-cf1b275cba83 None None] MFA Rules not processed for user `ced10e76e5cc4443819157c821f82226`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:18.660892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a2120eed-4f44-49d5-a893-cf1b275cba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:18.937393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 872/1749] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:19.875617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:19.876210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:19.876210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:19.878168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 878/1750] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:19.946655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:19.947878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:19.948396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:20.035513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] MFA Rules not processed for user `8a34be8446d14cf1b9a76e9ffd0d4921`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:20.039688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.188042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:20.190326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.233621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a053a-0af3-456e-8adb-23d2c4c54ade None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:20.245696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:20.246953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:20.247457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:20.247969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] PATH_INFO: `/v3/projects/4a91531b73624e06a99d8a425a9a5b83` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:20.253506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a91531b73624e06a99d8a425a9a5b83)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:20.256108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:20.280196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:20.302890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 873/1751] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:20.380580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:20.382212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.410727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a818924-fd8f-4bca-980d-cc1290744580 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:20.423712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a91531b73624e06a99d8a425a9a5b83', 'request_id': 'req-713fc4e0-6ee2-4630-8435-af126b5c91e1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.441749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a91531b73624e06a99d8a425a9a5b83', 'request_id': 'req-713fc4e0-6ee2-4630-8435-af126b5c91e1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.462645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:20.469827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a91531b73624e06a99d8a425a9a5b83', 'request_id': 'req-713fc4e0-6ee2-4630-8435-af126b5c91e1'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.488952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] The token cache is being invalidate because project 4a91531b73624e06a99d8a425a9a5b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:20.489242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.492176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.498108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 879/1752] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:20 2022] DELETE /identity/v3/projects/4a91531b73624e06a99d8a425a9a5b83 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:20.499862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:20.501259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.703054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:20.703765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:20.705323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:20.706024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:20.715453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.744062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:20.745736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:20.770385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 874/1753] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:20.771291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:20.775493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:20.784683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc4e0-6ee2-4630-8435-af126b5c91e1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:20.802943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:20.805548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:20.805861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:20.806409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] PATH_INFO: `/v3/users/c768cb426e264ef9bc382ada1c966444` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:20.818093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e8f7b-8cd2-4224-9512-a4bb45b88362 service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:20.818414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c768cb426e264ef9bc382ada1c966444)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:20.820593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:20.840335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:20.841525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:20.841808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:20.842091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] PATH_INFO: `/v3/users/1b2425d0a41e44a38b64aa1a2526b276` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:20.843339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2425d0a41e44a38b64aa1a2526b276)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:20.846182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:20.980376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c768cb426e264ef9bc382ada1c966444'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.996481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2425d0a41e44a38b64aa1a2526b276'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.996972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c768cb426e264ef9bc382ada1c966444'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:20.997318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c768cb426e264ef9bc382ada1c966444'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.026341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2425d0a41e44a38b64aa1a2526b276'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.028986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2425d0a41e44a38b64aa1a2526b276'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.030754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c768cb426e264ef9bc382ada1c966444'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.031614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9554612d-a9e6-41ce-8d40-f0adf237a68c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c768cb426e264ef9bc382ada1c966444'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.061439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 880/1754] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:20 2022] DELETE /identity/v3/users/c768cb426e264ef9bc382ada1c966444 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:21.092292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2425d0a41e44a38b64aa1a2526b276'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.115543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e9876fe5-f3d3-454d-bfdf-67006877c59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2425d0a41e44a38b64aa1a2526b276'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:21.144829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 875/1755] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:20 2022] DELETE /identity/v3/users/1b2425d0a41e44a38b64aa1a2526b276 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:21.362459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168af8-68f1-4ed5-9a98-0ebb71326188 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.362745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168af8-68f1-4ed5-9a98-0ebb71326188 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.363028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2168af8-68f1-4ed5-9a98-0ebb71326188 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:21.364896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 881/1756] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:21.384360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.384977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.385551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:21.418876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4117d-8a5d-42d6-9df0-a936691a7651 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.419167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4117d-8a5d-42d6-9df0-a936691a7651 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.419444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4117d-8a5d-42d6-9df0-a936691a7651 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:21.421950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 882/1757] 10.222.0.41 () {54 vars in 912 bytes} [Fri May 27 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:21.422301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:21.423550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:21.444062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.444389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.444672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:21.522340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:21.529174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:21.568339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:21.612326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 876/1758] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:21.712846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:21.719896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:21.728384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 883/1759] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:21.765972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:21.767431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:21.792658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4d62c-e2a5-4b2a-9ae0-7345541333fa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:21.798801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:21.800204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.800859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.801394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:21.818883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:21.820803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:21.824677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:21.850943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 884/1760] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:21.852101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:21.856881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:21.966917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15f49-3c69-4080-9f41-6e6875b347da None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:21.989098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:21.989997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:21.990282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:21.990559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] PATH_INFO: `/v3/projects/e932721cf3cc42b8bff32e55e6d485a6` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:22.001948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e932721cf3cc42b8bff32e55e6d485a6)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:22.008928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:22.019996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b82dda-6120-44a8-8fb0-6283653f2d27 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:22.043326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:22.044979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:22.045532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:22.046343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] PATH_INFO: `/v3/projects/267ba61d318543358dfc82753098ec7c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:22.048121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267ba61d318543358dfc82753098ec7c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:22.054648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:22.175557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267ba61d318543358dfc82753098ec7c', 'request_id': 'req-278b3ed1-9853-44ec-b36f-d6cd5489fc26'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.181641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e932721cf3cc42b8bff32e55e6d485a6', 'request_id': 'req-5a548a6e-60ba-4e31-850a-1d83e91f11db'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.207072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e932721cf3cc42b8bff32e55e6d485a6', 'request_id': 'req-5a548a6e-60ba-4e31-850a-1d83e91f11db'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.213027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267ba61d318543358dfc82753098ec7c', 'request_id': 'req-278b3ed1-9853-44ec-b36f-d6cd5489fc26'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.222211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e932721cf3cc42b8bff32e55e6d485a6', 'request_id': 'req-5a548a6e-60ba-4e31-850a-1d83e91f11db'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.244967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267ba61d318543358dfc82753098ec7c', 'request_id': 'req-278b3ed1-9853-44ec-b36f-d6cd5489fc26'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.262727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] The token cache is being invalidate because project e932721cf3cc42b8bff32e55e6d485a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:22.263297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.264079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.271998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 877/1761] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:21 2022] DELETE /identity/v3/projects/e932721cf3cc42b8bff32e55e6d485a6 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:22.271998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:22.277348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:22.325769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] The token cache is being invalidate because project 267ba61d318543358dfc82753098ec7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:22.328091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.332747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.343482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 885/1762] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:21 2022] DELETE /identity/v3/projects/267ba61d318543358dfc82753098ec7c => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:22.345876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:22.349921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:22.571325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548a6e-60ba-4e31-850a-1d83e91f11db admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:22.581940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:22.582758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:22.583044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:22.583403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:22.583944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3ed1-9853-44ec-b36f-d6cd5489fc26 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:22.585434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:22.640531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:22.696379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:22.783256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:22.784823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:22.785219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:22.785609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] PATH_INFO: `/v3/users/87a4e5257aa8443fb564a8427e672196` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:22.790438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a4e5257aa8443fb564a8427e672196)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:22.796050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:22.798785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:22.806189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:22.884596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 878/1763] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:22.886596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:22.889169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:22.938905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a4e5257aa8443fb564a8427e672196'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.974016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a4e5257aa8443fb564a8427e672196'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:22.975174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a4e5257aa8443fb564a8427e672196'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.020583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd673d9-a2da-4ca1-a9e8-15d193fb0f91 service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:23.063581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a4e5257aa8443fb564a8427e672196'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.066250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a4e5257aa8443fb564a8427e672196'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.074110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:23.079226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:23.079534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:23.079815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] PATH_INFO: `/v3/users/853955c46f604107bb55b587a94308ed` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:23.089330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=853955c46f604107bb55b587a94308ed)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:23.105673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:23.127152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 886/1764] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:22 2022] DELETE /identity/v3/users/87a4e5257aa8443fb564a8427e672196 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:23.127152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:23.127152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:23.302163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853955c46f604107bb55b587a94308ed'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.345460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853955c46f604107bb55b587a94308ed'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.346329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853955c46f604107bb55b587a94308ed'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.398936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853955c46f604107bb55b587a94308ed'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.410470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853955c46f604107bb55b587a94308ed'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.423495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 879/1765] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:22 2022] DELETE /identity/v3/users/853955c46f604107bb55b587a94308ed => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:23.434191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcba64-5761-4330-8df3-896c47517773 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:23.461379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:23.464131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:23.575257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:23.577851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:23.578137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:23.578410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] PATH_INFO: `/v3/projects/e5fd3a395a754024babe5389c65e54b7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:23.583472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fd3a395a754024babe5389c65e54b7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:23.589337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:23.595985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:23.709935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea0167-0437-4f17-96dc-f681776f9f3b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:23.724652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:23.725122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:23.725513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:23.727500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:23.733616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:23.775208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fd3a395a754024babe5389c65e54b7', 'request_id': 'req-51c60484-2592-46cf-aa40-a5ffb3a65bde'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.780437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:23.782297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b5383-24d5-4627-9512-b3deb15862ae service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:23.788354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fd3a395a754024babe5389c65e54b7', 'request_id': 'req-51c60484-2592-46cf-aa40-a5ffb3a65bde'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.807645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fd3a395a754024babe5389c65e54b7', 'request_id': 'req-51c60484-2592-46cf-aa40-a5ffb3a65bde'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.824457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] The token cache is being invalidate because project e5fd3a395a754024babe5389c65e54b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:23.824776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.825781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-51c60484-2592-46cf-aa40-a5ffb3a65bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:23.826781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 880/1766] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:23.849948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 887/1767] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:23 2022] DELETE /identity/v3/projects/e5fd3a395a754024babe5389c65e54b7 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:23.850585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:23.851038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:23.852033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:23.920223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:23.921483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:23.954869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:24.071550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 881/1768] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:24.080089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.080737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.080737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.152410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:24.156716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:24.195760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 888/1769] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:24.196915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:24.229647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad70b5-3018-44aa-bb44-8fe0f4520b01 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:24.248642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:24.249949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.252440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.252944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.253977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:24.255880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:24.261928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:24.317011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 882/1770] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:24.323749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:24.340693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d732a0-664a-4425-9e86-cb0ba6ca3ee9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:24.356902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:24.358041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.358602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.359104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.360495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:24.364809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:24.420623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 889/1771] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:24 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:24.426719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:24.478685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27b57b-4d25-455f-84a8-707b8853360b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:24.505690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:24.509016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:24.679126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31576130-eff1-43e4-9fcd-c4c6258fa196 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:24.694817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:24.696263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.696581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.696867 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] PATH_INFO: `/v3/projects/f7069066f7b541519f7f4ed54378adcf` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.699521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7069066f7b541519f7f4ed54378adcf)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:24.701542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:24.747472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:24.748692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.750745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.750745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.751749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:24.769999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca448e5-b309-4649-b0cd-d0d1ba9408cb admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:24.839234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 883/1772] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:24 2022] POST /identity/v3/users => generated 293 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:24.840661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04f6-f91a-44ad-abed-32348a686645 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:24.841235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04f6-f91a-44ad-abed-32348a686645 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:24.841659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04f6-f91a-44ad-abed-32348a686645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:24.843662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7069066f7b541519f7f4ed54378adcf', 'request_id': 'req-f4d289d8-a662-414d-8fe4-9d973e6684ac'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:24.887515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7069066f7b541519f7f4ed54378adcf', 'request_id': 'req-f4d289d8-a662-414d-8fe4-9d973e6684ac'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:24.916065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7069066f7b541519f7f4ed54378adcf', 'request_id': 'req-f4d289d8-a662-414d-8fe4-9d973e6684ac'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:24.925796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:24.936747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-433f04f6-f91a-44ad-abed-32348a686645 None None] MFA Rules not processed for user `d25dc63e8a4c4af8af2fe29b7819212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:24.938261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-433f04f6-f91a-44ad-abed-32348a686645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:24.953645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] The token cache is being invalidate because project f7069066f7b541519f7f4ed54378adcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:24.954143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:24.956803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:24.960851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 890/1773] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:24 2022] DELETE /identity/v3/projects/f7069066f7b541519f7f4ed54378adcf => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:24.961858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:24.966332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:25.121659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 884/1774] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:25.121659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f417-bfba-485f-bff9-b6adb9f6c4a4 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.122267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f417-bfba-485f-bff9-b6adb9f6c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.122267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0f417-bfba-485f-bff9-b6adb9f6c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.198848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d289d8-a662-414d-8fe4-9d973e6684ac admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.217836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:25.221743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.223535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.223931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.229693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:25.234221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:25.245637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-34c0f417-bfba-485f-bff9-b6adb9f6c4a4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:25.246944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-34c0f417-bfba-485f-bff9-b6adb9f6c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:25.272189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 891/1775] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:25.274989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:25.294770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 885/1776] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:25.297651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.297947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.298309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.326597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe2455-b5cb-4dc5-851b-ce20af97526d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.354290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:25.356571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.357595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:25.358155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.358842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:25.359273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.361687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:25.368888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:25.393827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 886/1777] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:25.395908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:25.423954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0232a26-5285-4afe-bdf8-1363a4250b27 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.441839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:25.442473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.442950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.443258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.447424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 892/1778] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:25.447617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:25.447617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:25.449747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:25.461186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:25.474642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe28500-9af5-465d-ba03-ad504a6e13ef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.484304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:25.488184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.491134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.491615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] PATH_INFO: `/v3/projects/55823005c36d4eda881a53197432ad20/users/999da3d87d9845818ff582e60e9c51a6/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.503099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 887/1779] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:25.507754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:25.534319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3bd2-b26b-46ee-b59a-115209a2060b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.545337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55823005c36d4eda881a53197432ad20, user_id=999da3d87d9845818ff582e60e9c51a6, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:25.548263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:25.549001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.549450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.549862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.550741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:25.552865 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7d80-1161-4a5d-9cfb-9ea2bef23552 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:25.561603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:25.580623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:25.597517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 893/1780] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:25 2022] PUT /identity/v3/projects/55823005c36d4eda881a53197432ad20/users/999da3d87d9845818ff582e60e9c51a6/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:25.599679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:25.603820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:25.630249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 888/1781] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:25 2022] POST /identity/v3/projects => generated 409 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:25.649917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:25.650251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:25.650512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:25.738071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] MFA Rules not processed for user `999da3d87d9845818ff582e60e9c51a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:25.740819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:25.846481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4929fb4-ae86-48fd-8b95-aa2f3c1274c6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:25.941789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 889/1782] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:25.942890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.006758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.007695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.008175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.008477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] PATH_INFO: `/v3/projects/ae474d6b6ed94af987351e90515ba87c` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.014107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae474d6b6ed94af987351e90515ba87c)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.018495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.028537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:26.051633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b717bb-7648-484f-a9e1-878757909d41 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.065226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.066291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.066457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.066842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.067740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.069898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.123018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 890/1783] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:25 2022] POST /identity/v3/users => generated 305 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:26.124185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.138476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5e202-920c-4f13-afde-7fc1a369545b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.139682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae474d6b6ed94af987351e90515ba87c', 'request_id': 'req-bc07d815-00af-4854-8313-75dea2c8df95'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:26.156638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae474d6b6ed94af987351e90515ba87c', 'request_id': 'req-bc07d815-00af-4854-8313-75dea2c8df95'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:26.171697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae474d6b6ed94af987351e90515ba87c', 'request_id': 'req-bc07d815-00af-4854-8313-75dea2c8df95'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:26.196906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] The token cache is being invalidate because project ae474d6b6ed94af987351e90515ba87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:26.197218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:26.198937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:26.205967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 894/1784] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:25 2022] DELETE /identity/v3/projects/ae474d6b6ed94af987351e90515ba87c => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:26.207009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.214777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:26.287239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.287933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.288219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.288609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.292223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.297312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267231b-f43b-4521-bd71-811287fa2021 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.339667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:26.368887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 891/1785] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:26 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:26.372297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.373133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.374201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.384345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d815-00af-4854-8313-75dea2c8df95 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.457082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:26.462378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:26.570608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.571458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.572166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.572583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.573451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.575228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.597197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 895/1786] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:26.603252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.606497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 892/1787] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:26.607529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.608587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:26.626480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edeee35-7b76-499e-9eef-b884c99d7500 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.650079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.652598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.653106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.653210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.654010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.654628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9816a3e-a0a0-4199-8d4f-f2a49bc7195c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.664754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69228c11-2cee-44d5-a1bf-506486c304fd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.669747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.670517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.672940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.673358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.674211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.681670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.689701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 893/1788] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:26.695602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.697935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.698233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.726543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:26.728392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:26.743440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 894/1789] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:26.743824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 896/1790] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:26 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:26.743937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.744607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.758892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae6ac2-5247-4d93-9b39-3b3ad02f25de None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.763735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7249d6-bba6-4255-96fe-7fb5cdaa86f8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.771612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.772949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.773840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.773840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] PATH_INFO: `/v3/projects/dd496677ea7141b3acaca1bc7cbc69b0/users/e3b62f15b23a4d5285a39d1086326ff3/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.775605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:26.776304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:26.776621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:26.776903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:26.779548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.781330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.800584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd496677ea7141b3acaca1bc7cbc69b0, user_id=e3b62f15b23a4d5285a39d1086326ff3, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:26.802568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 897/1791] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:26.803740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.805852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:26.824981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 895/1792] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:26 2022] PUT /identity/v3/projects/dd496677ea7141b3acaca1bc7cbc69b0/users/e3b62f15b23a4d5285a39d1086326ff3/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:26.826952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:26.832471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34310df3-baf2-4b02-bf6f-8d0def9a3d72 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:26.842115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:27.033734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.034524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.034924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.035283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.036218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:27.036650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.038379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85f979-6b75-43d3-afa2-bccfbea4a41a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.047743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f058d4b-5bd8-4e1e-a962-cbdd3f8ba956 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.069565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 898/1793] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:27.071657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.075344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.075814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.076125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.076228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.076416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.088011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] PATH_INFO: `/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75/users/fda36868a86c4937afc6eb9487bb060e/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.138350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] MFA Rules not processed for user `e3b62f15b23a4d5285a39d1086326ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:27.139897 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:27.164838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ef220fd7ae4d8f81c8b8c8913a4c75, user_id=fda36868a86c4937afc6eb9487bb060e, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.180786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.236500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 896/1794] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:26 2022] PUT /identity/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75/users/fda36868a86c4937afc6eb9487bb060e/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:27.243459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.312013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 899/1795] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:27.313670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.335476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75730a35-ca6e-40df-96aa-290fcc8b4dda None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.373062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:27.451505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fef94-75d5-4dc2-bfec-fe9f91ba8c54 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.468112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.470545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.473051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.473392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.475233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.481399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.513233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.516610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.517091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.519218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.520127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.528894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.560426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 897/1796] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:27 2022] POST /identity/v3/projects => generated 387 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:27.562888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.568170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 900/1797] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:27.568170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.593643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63a09f-579a-4cad-aba9-7d924698ec6d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.642811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:27.732394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.736061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.736400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.737372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.738914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.746086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.761261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c209e-88f6-4bb5-b505-763b93fb2289 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.770809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.772652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.773680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.774125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.775478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.779838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aa724-acce-4ad6-a9aa-94f3a669e5ee admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.795474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 901/1798] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:27 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:27.796676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.809678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af50ff-0c12-4e0a-8a14-55bbdd5bd92d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:27.815692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:27.816923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.817295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.817684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] PATH_INFO: `/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75/users/fda36868a86c4937afc6eb9487bb060e/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.821044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ef220fd7ae4d8f81c8b8c8913a4c75, user_id=fda36868a86c4937afc6eb9487bb060e, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:27.826438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:27.848622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 902/1799] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:27 2022] PUT /identity/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75/users/fda36868a86c4937afc6eb9487bb060e/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:27.850063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:27.872732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 898/1800] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:27 2022] POST /identity/v3/projects => generated 409 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:27.872732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:27.874221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:27.874518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:27.956647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] MFA Rules not processed for user `fda36868a86c4937afc6eb9487bb060e`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:27.958018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:28.072312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac503-627b-425d-ad6d-e1a4b5f4adea admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.153633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 899/1801] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:28.154970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.200796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.201564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.201898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.202179 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.202959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.208898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.224282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:28.225559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 903/1802] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:28.227044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.253481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:28.285967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673ffc0-2ecc-4b8f-8848-2b4e976db165 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.292821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.295313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.295622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.295908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.296662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.298775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.332796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 900/1803] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:28 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:28.333902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.354367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc871c06-dd28-47f5-8a33-9db4e2336b01 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.363027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.363927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.364220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.364554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.365420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.369248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.394023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:28.405848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8747174-3bf6-4019-aa6c-c4c830f5368d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.406055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 901/1804] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:28.412391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.422789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.423656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.424086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.424534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.427143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aac5c6-d347-4d52-ad3e-a510181284af admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.434676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.435533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.435823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.437088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.437975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.439764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.442939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.446564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.459175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 902/1805] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:28.459175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.468633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 904/1806] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:28.469739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.503692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7181b-1756-438f-a1bb-4a107ccd5be9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.510209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02c34-c6ae-46ad-ad35-1fa7182f5b85 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.523354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.525919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.528038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.529426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] PATH_INFO: `/v3/projects/946496e8a67a4ced8c4efe81a1b4911a/users/e6914ab7bed84c92837888d8db037b84/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.536842 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.537995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.540767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.543030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] PATH_INFO: `/v3/projects/5c198a961c7c4e809578d2b156216b6d/users/0820b46b177f497cac5f1bf0e9a73248/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.573314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946496e8a67a4ced8c4efe81a1b4911a, user_id=e6914ab7bed84c92837888d8db037b84, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.576678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.581224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c198a961c7c4e809578d2b156216b6d, user_id=0820b46b177f497cac5f1bf0e9a73248, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.583921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3377c6-b23d-4999-8702-707d66b5ac6b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.604102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 905/1807] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:28 2022] PUT /identity/v3/projects/946496e8a67a4ced8c4efe81a1b4911a/users/e6914ab7bed84c92837888d8db037b84/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:28.607372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 903/1808] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:28 2022] PUT /identity/v3/projects/5c198a961c7c4e809578d2b156216b6d/users/0820b46b177f497cac5f1bf0e9a73248/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:28.608292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.622797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.625167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.625707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.729774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] MFA Rules not processed for user `0820b46b177f497cac5f1bf0e9a73248`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:28.734688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:28.782701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3061-4f7b-4dfa-803a-e55124789068 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.788209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.790135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.790488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.790778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.791521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.793417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.813055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 906/1809] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:28.817734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.830880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a5b16-476c-468b-b82f-cea25c1eac44 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.838081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:28.839534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.839958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.840349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] PATH_INFO: `/v3/projects/946496e8a67a4ced8c4efe81a1b4911a/users/e6914ab7bed84c92837888d8db037b84/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.842875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946496e8a67a4ced8c4efe81a1b4911a, user_id=e6914ab7bed84c92837888d8db037b84, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:28.847747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abe5e2-da24-46b0-9362-af870d3cba91 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:28.872776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 907/1810] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:28 2022] PUT /identity/v3/projects/946496e8a67a4ced8c4efe81a1b4911a/users/e6914ab7bed84c92837888d8db037b84/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:28.884892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:28.885360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:28.885798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:28.913166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 904/1811] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:28.918332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:28.943554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be5d8f-c574-49f0-88b1-7b4af826edcc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:28.960365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] MFA Rules not processed for user `e6914ab7bed84c92837888d8db037b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:28.963336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:29.072148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.073681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.074133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.074845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.076372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.079087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.097713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:29.128967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 905/1812] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:28 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:29.132731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:29.201019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 908/1813] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:29.241948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:29.244237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:29.307358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492913f7-c51f-4624-8198-ca58bdcc2301 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:29.311591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:29.327216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.328218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.329810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.331855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.332801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.337579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.355446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b883aa4-443a-451f-aa35-0b1c5ebbfa42 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:29.392434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 906/1814] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:29 2022] POST /identity/v3/users => generated 294 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:29.393547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:29.427954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95aaa99-ebe6-451a-a285-872f26fa75ad admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:29.442441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.445749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.447730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.448214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.453189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.457664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.478210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.479393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.479831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.480147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.501078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 907/1815] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:29.502794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:29.525624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc408a2-a563-499c-bdea-c055610286ee admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:29.531650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.532407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.534804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.535274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.536069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.538240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.579143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 908/1816] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:29.580158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:29.611776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c84e3f-53ab-4761-a9ef-f8dfaf112024 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:29.632863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:29.633400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.636729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.637043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] PATH_INFO: `/v3/projects/394c06bc85e04d21b060b2abb5379594/users/236069ada307444498e02d2b37fd2181/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.696999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.700149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8e7b5-d1b1-4269-b165-32a3138b8a69 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.706831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=394c06bc85e04d21b060b2abb5379594, user_id=236069ada307444498e02d2b37fd2181, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:29.716435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244131c9-7c1c-40bc-8008-f23e65bb7d7e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:29.739921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 909/1817] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:29.743464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3d86-47c4-4c27-87fb-72b42fda37f4 None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.744138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3d86-47c4-4c27-87fb-72b42fda37f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.744406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d3d86-47c4-4c27-87fb-72b42fda37f4 None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.766409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 910/1818] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:12:29 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:29.770892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bff6f-0a5e-4cfe-9182-5885a491ca8e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.771182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bff6f-0a5e-4cfe-9182-5885a491ca8e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.775970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bff6f-0a5e-4cfe-9182-5885a491ca8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.781346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 909/1819] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:29 2022] PUT /identity/v3/projects/394c06bc85e04d21b060b2abb5379594/users/236069ada307444498e02d2b37fd2181/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:29.793630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:29.793914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:29.794469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:29.850130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2a7bff6f-0a5e-4cfe-9182-5885a491ca8e None None] MFA Rules not processed for user `ced10e76e5cc4443819157c821f82226`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:29.859268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2a7bff6f-0a5e-4cfe-9182-5885a491ca8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:29.874306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:29.885834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:30.047006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 911/1820] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:30.047901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.048600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.048893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.092743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] MFA Rules not processed for user `236069ada307444498e02d2b37fd2181`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:30.095003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:30.137247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 910/1821] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:12:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:30.154277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.168136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:30.211961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:30.249965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc446a-5aed-4017-bd4c-ba781c5c7e2f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:30.285387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:30.286240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.286448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.286737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.323840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 912/1822] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:30.326218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.329972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:30.331619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:30.386511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 911/1823] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:30.422588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.475476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:30.542910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507cd464-96d5-4bf3-a613-fea0e457b933 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:30.557417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:30.558139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.558423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.560012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.560861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:30.564888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:30.634659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 913/1824] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:30 2022] POST /identity/v3/projects => generated 387 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:30.640548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.646220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3cac6-576e-42aa-9367-86694abd1d8c service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:30.800794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:30.801531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.801822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.802102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.855250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690839-42a4-4a4b-98c1-9bafa932dc72 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:30.885045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:30.885746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.886032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.886312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.890819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:30.893846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:30.923532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:30.929962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:30.936937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 914/1825] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:30 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:30.937988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.959621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b503ec-f1a1-44e4-b5b3-b77402ef0033 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:30.960805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 912/1826] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:30.963930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:30.965465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:30.967210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:30.967397 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:30.967826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] PATH_INFO: `/v3/users/999da3d87d9845818ff582e60e9c51a6` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:30.970299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] RBAC: Authorizing `identity:delete_user(user_id=999da3d87d9845818ff582e60e9c51a6)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:30.972309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:30.998926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b016b-5e23-4817-ba95-de38a56779a7 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.013659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.014416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.014735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.016629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.017541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.021587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.035505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 913/1827] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:31.045875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:31.057110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:31.061533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999da3d87d9845818ff582e60e9c51a6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:31.061941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330555-5709-432e-a5ac-2ef1ae894cf4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.074794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999da3d87d9845818ff582e60e9c51a6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:31.075765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.076007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999da3d87d9845818ff582e60e9c51a6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:31.076752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.077039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.077320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] PATH_INFO: `/v3/projects/cf0f691f83694964b6c432670ca59fff/users/eb500bf7a7ba413bb9cf72ca3ea41db1/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.097284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999da3d87d9845818ff582e60e9c51a6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:31.098352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6e596ed7-dad5-4f77-9e64-b5ea2d64825d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999da3d87d9845818ff582e60e9c51a6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:31.109430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f691f83694964b6c432670ca59fff, user_id=eb500bf7a7ba413bb9cf72ca3ea41db1, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.112488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.119839 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 915/1828] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:30 2022] DELETE /identity/v3/users/999da3d87d9845818ff582e60e9c51a6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:31.123849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.123849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.124296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.144090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 914/1829] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:31 2022] PUT /identity/v3/projects/cf0f691f83694964b6c432670ca59fff/users/eb500bf7a7ba413bb9cf72ca3ea41db1/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:31.150153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:31.153103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:31.169484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:31.171292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:31.290979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:31.300833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 916/1830] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:31.327438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:31.329513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:31.340275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ac2c8-eb72-4693-8aeb-809911beb787 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.347273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.348014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.348301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.348607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.349835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.353092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.379116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 915/1831] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:31.380339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:31.393565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746aea81-0ce9-43a7-b0c6-e5656d6f4da1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.408048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.409264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.410002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.410434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] PATH_INFO: `/v3/projects/cf0f691f83694964b6c432670ca59fff/users/eb500bf7a7ba413bb9cf72ca3ea41db1/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.412858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f691f83694964b6c432670ca59fff, user_id=eb500bf7a7ba413bb9cf72ca3ea41db1, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.416340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327f96-1371-4637-bec8-778f3f6cfa4f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.449638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 916/1832] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:31 2022] PUT /identity/v3/projects/cf0f691f83694964b6c432670ca59fff/users/eb500bf7a7ba413bb9cf72ca3ea41db1/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:31.451970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.452434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.452946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.494719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] MFA Rules not processed for user `eb500bf7a7ba413bb9cf72ca3ea41db1`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:31.496145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:31.566796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c34add-55b4-4899-93ef-4b83716e3ee0 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.649307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 917/1833] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:31.661702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:31.663137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:31.712430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.713322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.713642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.716753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.790525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:31.800323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db776690-13eb-4a3c-9cb6-dcb56abb1d2b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:31.824216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:31.826474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:31.827958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:31.828495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:31.853488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.855589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.876568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 918/1834] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:31.885295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:31.886669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:31.904944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 917/1835] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:32.893098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:32.894669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:32.917618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d20076-9744-437b-8075-c6e73ed61ea2 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:32.922785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:32.923584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:32.923941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:32.924752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:32.928284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:33.029021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:33.030695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:33.035168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:33.043360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 919/1836] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:33.441498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:33.442996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:33.476284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0931b-55f0-491a-8e94-efc41c9f3c18 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:33.483930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:33.485434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:33.485804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:33.486255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] PATH_INFO: `/v3/projects/55823005c36d4eda881a53197432ad20` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:33.487823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=55823005c36d4eda881a53197432ad20)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:33.490989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:33.579652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55823005c36d4eda881a53197432ad20', 'request_id': 'req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.592112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55823005c36d4eda881a53197432ad20', 'request_id': 'req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.603751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55823005c36d4eda881a53197432ad20', 'request_id': 'req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.637215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] The token cache is being invalidate because project 55823005c36d4eda881a53197432ad20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:33.638213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.639969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1b7cee5-26cd-46b3-869f-b5caf4f7ce9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.643841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 918/1837] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:33 2022] DELETE /identity/v3/projects/55823005c36d4eda881a53197432ad20 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:33.648068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:33.667636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5a8dc-4728-4579-bc9b-dbeb1b1a6efe service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:33.766034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:33.767139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:33.767483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:33.767910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] PATH_INFO: `/v3/users/fda36868a86c4937afc6eb9487bb060e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:33.769954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda36868a86c4937afc6eb9487bb060e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:33.776517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:33.839443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda36868a86c4937afc6eb9487bb060e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.847552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda36868a86c4937afc6eb9487bb060e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.849948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda36868a86c4937afc6eb9487bb060e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.869647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda36868a86c4937afc6eb9487bb060e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.879870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-337c3ff4-6fe7-49c1-a73d-dc2fae06b3ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda36868a86c4937afc6eb9487bb060e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:33.888793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 920/1838] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:33 2022] DELETE /identity/v3/users/fda36868a86c4937afc6eb9487bb060e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:34.225308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.226403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.226795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.228641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 919/1839] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:34.245175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.245703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.245984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.269363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:34.270683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:34.331094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.332967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:34.365132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 921/1840] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:34.396721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.399011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:34.421931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da0783-52da-4894-a0a5-0971aabf4f0b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.428148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.428836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.429281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.429427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.437146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:34.439807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.441928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37861d-d4b5-42a1-b8eb-1f1d3498510a service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.447284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3448a1-e4a8-481d-88ff-20164d1ea24c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.452064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.452967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.453253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.453525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] PATH_INFO: `/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.454714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ef220fd7ae4d8f81c8b8c8913a4c75)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.455394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 922/1841] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:34.456710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.508425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ef220fd7ae4d8f81c8b8c8913a4c75', 'request_id': 'req-17af16c3-c044-4826-898e-4fe2f75b9468'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:34.510516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:34.518940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ef220fd7ae4d8f81c8b8c8913a4c75', 'request_id': 'req-17af16c3-c044-4826-898e-4fe2f75b9468'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:34.528883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ef220fd7ae4d8f81c8b8c8913a4c75', 'request_id': 'req-17af16c3-c044-4826-898e-4fe2f75b9468'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:34.541009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] The token cache is being invalidate because project d9ef220fd7ae4d8f81c8b8c8913a4c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:34.541266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:34.541960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-17af16c3-c044-4826-898e-4fe2f75b9468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:34.543550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 920/1842] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:34 2022] DELETE /identity/v3/projects/d9ef220fd7ae4d8f81c8b8c8913a4c75 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:34.548327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.548654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.548939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.567548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:34.568786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:34.637877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 923/1843] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:34.645761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.646058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.646345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.665655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:34.667001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:34.677593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 921/1844] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:34.682976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.692804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf0af9-e82e-4cd7-8161-f29d0885a52b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.698872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.699542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.699826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.700103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.700870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:34.701182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.702781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.719151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 924/1845] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:34.721889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.735902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651c375-0777-43d8-a9b0-141b0ab6e47e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.740775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.742341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.742938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.743347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.744136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.746181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.785518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 922/1846] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:34 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:34.786967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.801448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1854-6191-4d7a-b7af-afd5312fcf19 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.888942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.889614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.889895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.890169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.890854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.892797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.914631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 925/1847] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:34 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:34.919026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.930490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24b4ab-c2de-4c17-a16c-9899368a1a7b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.935130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.935783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.936064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.936336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.937084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.938682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.949709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 923/1848] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:34.953709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.964656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f79f1-a97e-47b2-9aff-54ff194d12d4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:34.969728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:34.970388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:34.970675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:34.970956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:34.971733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:34.973362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fabaa3-6688-408f-a040-b2f5fa80ec9c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:34.983626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 926/1849] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:34.987669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:34.997749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0797-9fc0-4ada-9abb-a9ba9d6e897d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.002343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.003216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.003499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.003772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] PATH_INFO: `/v3/projects/db9b86e4dd344b58b0cf7d1436650729/users/bfd4138aa1004c82be626843c50c91a2/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.019293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db9b86e4dd344b58b0cf7d1436650729, user_id=bfd4138aa1004c82be626843c50c91a2, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.021991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.035465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 924/1850] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:34 2022] PUT /identity/v3/projects/db9b86e4dd344b58b0cf7d1436650729/users/bfd4138aa1004c82be626843c50c91a2/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:35.047109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.047326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.047609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.076276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] MFA Rules not processed for user `bfd4138aa1004c82be626843c50c91a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:35.077524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:35.133289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:35.153205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 927/1851] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:35.157663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.223576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa48855-ab28-4e22-8bc3-dccdb2e876f9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.229244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.229900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.230182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.230458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.231147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.233118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.258351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 925/1852] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:35 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:35.262381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.273550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec223-504d-4baf-a8a6-da0b7944567d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.328971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.329639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.329926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.330208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.330896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.332938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.353849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 928/1853] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:35 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:35.358055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.368408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dbc61-ebbd-4c36-b5b1-1b0c0333294c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.373358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.374026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.374341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.374619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.375341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.377006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.388221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 926/1854] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:35.392325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.403181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d7179-63a8-46b9-ba6c-b01975a39e4b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.407875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.408787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.409077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.409360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] PATH_INFO: `/v3/projects/66daae71082d4c32b2059c2c1e0f82eb/users/56e63dfdb87b4f68a36ad96c898bcd7a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.425796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=66daae71082d4c32b2059c2c1e0f82eb, user_id=56e63dfdb87b4f68a36ad96c898bcd7a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.428545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.442524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 929/1855] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:35 2022] PUT /identity/v3/projects/66daae71082d4c32b2059c2c1e0f82eb/users/56e63dfdb87b4f68a36ad96c898bcd7a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:35.447330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.458003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f5ce5-cff2-48b5-8429-131b219cf328 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.508410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:35.520493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.521166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.521485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.521772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.522546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.525064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc24c36-5521-4d1a-9640-c8b0d0cf3674 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.534883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 927/1856] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:35.539411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.549819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc44b2-a856-41c6-8170-e53dcff790ef admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.554745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.555644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.555953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.556255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] PATH_INFO: `/v3/projects/66daae71082d4c32b2059c2c1e0f82eb/users/56e63dfdb87b4f68a36ad96c898bcd7a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.558266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66daae71082d4c32b2059c2c1e0f82eb, user_id=56e63dfdb87b4f68a36ad96c898bcd7a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.560951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.571719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 930/1857] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:35 2022] PUT /identity/v3/projects/66daae71082d4c32b2059c2c1e0f82eb/users/56e63dfdb87b4f68a36ad96c898bcd7a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:35.581522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.581755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.582034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.608857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] MFA Rules not processed for user `56e63dfdb87b4f68a36ad96c898bcd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:35.610052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:35.676373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 928/1858] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:35.688758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.690416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:35.746667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a347a9-9736-4575-b6b1-4704aefc81b6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.750275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.750906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.751188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.751463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.761013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.762426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.771734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 931/1859] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:35.858720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:35.869009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f89496-3024-42cf-8638-c17edfd7f265 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:35.872745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:35.873396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:35.873681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:35.873964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:35.956264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:35.957851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69293e7-b73a-452d-b5e5-a73cf13e9c0f service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:35.969111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 929/1860] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:36.354128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:36.444189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:36.488946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fb18f-26da-4456-b62d-74014c99e3b3 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:36.499929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:36.501469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:36.501819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:36.502163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] PATH_INFO: `/v3/users/bfd4138aa1004c82be626843c50c91a2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:36.503587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd4138aa1004c82be626843c50c91a2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:36.506373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:36.584921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd4138aa1004c82be626843c50c91a2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:36.595350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd4138aa1004c82be626843c50c91a2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:36.597510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd4138aa1004c82be626843c50c91a2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:36.638705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd4138aa1004c82be626843c50c91a2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:36.648859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd4138aa1004c82be626843c50c91a2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:36.660489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 932/1861] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:36 2022] DELETE /identity/v3/users/bfd4138aa1004c82be626843c50c91a2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:36.662220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:36.662813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:36.662813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:36.680508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:36.681724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:36.759680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 930/1862] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:36.792265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:36.793753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:36.948884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a676b76-f34b-4d1c-8075-30ff8359bc48 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:36.970859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:36.971698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:36.972080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:36.972506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:36.999600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:37.001717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:37.030329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 933/1863] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:38.441567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:38.443647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:38.470812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca8908-06b8-4fdd-8933-b8053d5277b6 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:38.483287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:38.484421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.485070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.485723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] PATH_INFO: `/v3/projects/db9b86e4dd344b58b0cf7d1436650729` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.492811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db9b86e4dd344b58b0cf7d1436650729)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:38.496018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:38.508319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:38.615725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db9b86e4dd344b58b0cf7d1436650729', 'request_id': 'req-bd263f76-51ed-481b-8738-30f9bae27dfc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.649567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db9b86e4dd344b58b0cf7d1436650729', 'request_id': 'req-bd263f76-51ed-481b-8738-30f9bae27dfc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.666817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db9b86e4dd344b58b0cf7d1436650729', 'request_id': 'req-bd263f76-51ed-481b-8738-30f9bae27dfc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.692523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] The token cache is being invalidate because project db9b86e4dd344b58b0cf7d1436650729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:38.694751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.697223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.703293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 931/1864] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:38 2022] DELETE /identity/v3/projects/db9b86e4dd344b58b0cf7d1436650729 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:38.707397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:38.721078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de2025-ee57-4520-9e43-1b05dfb2aaf9 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:38.831835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:38.833009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.833299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.833613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] PATH_INFO: `/v3/users/56e63dfdb87b4f68a36ad96c898bcd7a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.835496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e63dfdb87b4f68a36ad96c898bcd7a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:38.838163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:38.859246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:38.970645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e63dfdb87b4f68a36ad96c898bcd7a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.991014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e63dfdb87b4f68a36ad96c898bcd7a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:38.993630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e63dfdb87b4f68a36ad96c898bcd7a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.025329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e63dfdb87b4f68a36ad96c898bcd7a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.037089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e63dfdb87b4f68a36ad96c898bcd7a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.047545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 934/1865] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:38 2022] DELETE /identity/v3/users/56e63dfdb87b4f68a36ad96c898bcd7a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:39.389769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.391370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.398807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.400567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.412599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263f76-51ed-481b-8738-30f9bae27dfc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.510596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51bced-b6d1-428e-9f89-94345d95d129 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.513605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.515052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.515757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.516212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] PATH_INFO: `/v3/users/e3b62f15b23a4d5285a39d1086326ff3` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.516331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.517250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.517539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.517902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] PATH_INFO: `/v3/projects/66daae71082d4c32b2059c2c1e0f82eb` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.518821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b62f15b23a4d5285a39d1086326ff3)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.519379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66daae71082d4c32b2059c2c1e0f82eb)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.521260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:39.521425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:39.591040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66daae71082d4c32b2059c2c1e0f82eb', 'request_id': 'req-0d51eacb-04d5-42a3-8e18-3e1fd07df280'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.594355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b62f15b23a4d5285a39d1086326ff3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.598813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66daae71082d4c32b2059c2c1e0f82eb', 'request_id': 'req-0d51eacb-04d5-42a3-8e18-3e1fd07df280'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.603671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b62f15b23a4d5285a39d1086326ff3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.604887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b62f15b23a4d5285a39d1086326ff3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.607070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66daae71082d4c32b2059c2c1e0f82eb', 'request_id': 'req-0d51eacb-04d5-42a3-8e18-3e1fd07df280'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.619785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b62f15b23a4d5285a39d1086326ff3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.620634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5047918c-b4bb-4f83-8487-2336fd5da886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b62f15b23a4d5285a39d1086326ff3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.622350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] The token cache is being invalidate because project 66daae71082d4c32b2059c2c1e0f82eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:39.622774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.623497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0d51eacb-04d5-42a3-8e18-3e1fd07df280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:39.625291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 935/1866] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:39 2022] DELETE /identity/v3/projects/66daae71082d4c32b2059c2c1e0f82eb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:39.630116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb0ca-791a-4e46-af6e-9097f5251c9f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.630834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb0ca-791a-4e46-af6e-9097f5251c9f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.631159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb0ca-791a-4e46-af6e-9097f5251c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.636268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 932/1867] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:39 2022] DELETE /identity/v3/users/e3b62f15b23a4d5285a39d1086326ff3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:39.641670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.642280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.642659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.656794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-834cb0ca-791a-4e46-af6e-9097f5251c9f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:39.658640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-834cb0ca-791a-4e46-af6e-9097f5251c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.663082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:39.665424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.672278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:39.673900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:39.739239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 936/1868] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:39.746710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 933/1869] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:39.750536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.751123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.751716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.754470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.756061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.778809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:39.783453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.794024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 937/1870] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:39.798875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.811053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d985f-d6bf-451f-9f7c-14399ba24a2d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.816105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.816822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.817120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.817394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.818208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:39.818520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.820619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:39.831207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f2fd-1538-4448-bab9-e0fbdb94ed90 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.839601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 938/1871] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:39.839789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.839789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.839789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.839789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.841005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.850082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.851609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2a51-1e68-43bc-b73f-c480d299323d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.852368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:39.856420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.857105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.857394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.857674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.858372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.860285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:39.864791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 934/1872] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:39.890262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 939/1873] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:39 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:39.894193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.907504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9956da-b679-4946-9f9b-ef89e0366034 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:39.954841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:39.956715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:39.993555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:39.994325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:39.994625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:39.994930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:39.995643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:39.998072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.026533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 935/1874] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:39 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:40.030451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:40.042749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c4eb7-3e29-43f0-8702-c930ce9abab4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.049969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.051084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.051442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.051727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.052467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.055846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.072020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 936/1875] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:40.073012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de682-37e1-4b4a-8917-8c089b9e4ace admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.073143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:40.078006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.078855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.079537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.079895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.082450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:40.084398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772926c3-b2ee-4d34-9040-180c3b5bc717 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.090264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.091195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.091473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.091895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.092795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.094721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.109808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 937/1876] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:40.123652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:40.144918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25a461-56b3-4447-ab01-3c8b62a331fb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.160068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.161980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.164187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.165417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] PATH_INFO: `/v3/projects/7a4f2609a59542f9b6c6b314bf188f0e/users/55bfc6f311d241e69e687efaebe1ea24/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.203649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4f2609a59542f9b6c6b314bf188f0e, user_id=55bfc6f311d241e69e687efaebe1ea24, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.212271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfaae9-f0b0-49b2-8986-a1a83e730304 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.230893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:40.237757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.239743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.252367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 938/1877] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:40 2022] PUT /identity/v3/projects/7a4f2609a59542f9b6c6b314bf188f0e/users/55bfc6f311d241e69e687efaebe1ea24/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:40.254065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.255497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.255784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.298347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 940/1878] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:40.352772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] MFA Rules not processed for user `55bfc6f311d241e69e687efaebe1ea24`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:40.354593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:40.514262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 939/1879] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:40.607005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:40.608430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:40.630518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398fbac-746b-47fc-9d9f-be7c1467e655 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.684814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:40.723316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.724396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.724840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.725207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.737657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:40.739957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:40.744960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.749308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:40.762933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 941/1880] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:40.846663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c6f9c-3e68-4ff7-bc4a-f70033b82da5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:40.853403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:40.854799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:40.855146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:40.855429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] PATH_INFO: `/v3/projects/dd496677ea7141b3acaca1bc7cbc69b0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:40.856917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd496677ea7141b3acaca1bc7cbc69b0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:40.859161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:41.005051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd496677ea7141b3acaca1bc7cbc69b0', 'request_id': 'req-b165f242-503f-497d-89c1-8c2cd67d176a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.033520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd496677ea7141b3acaca1bc7cbc69b0', 'request_id': 'req-b165f242-503f-497d-89c1-8c2cd67d176a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.059573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd496677ea7141b3acaca1bc7cbc69b0', 'request_id': 'req-b165f242-503f-497d-89c1-8c2cd67d176a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.082071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] The token cache is being invalidate because project dd496677ea7141b3acaca1bc7cbc69b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:41.082382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.083165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.086167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 940/1881] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:40 2022] DELETE /identity/v3/projects/dd496677ea7141b3acaca1bc7cbc69b0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:41.090066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:41.106801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15509fb-43e5-4c46-ae3e-5fc0ce9c56c6 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:41.225143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:41.226183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:41.226352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:41.226830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] PATH_INFO: `/v3/users/e6914ab7bed84c92837888d8db037b84` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:41.238592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6914ab7bed84c92837888d8db037b84)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:41.243360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:41.333142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6914ab7bed84c92837888d8db037b84'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.342873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6914ab7bed84c92837888d8db037b84'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.344032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6914ab7bed84c92837888d8db037b84'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.359778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6914ab7bed84c92837888d8db037b84'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.370345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-371721c4-0e92-4c20-8ec8-45d5c7566eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6914ab7bed84c92837888d8db037b84'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:41.387944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 942/1882] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:41 2022] DELETE /identity/v3/users/e6914ab7bed84c92837888d8db037b84 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:41.898040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:41.900285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:41.940680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f242-503f-497d-89c1-8c2cd67d176a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:41.970467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:42.045665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.046909 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.047296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.047709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] PATH_INFO: `/v3/projects/946496e8a67a4ced8c4efe81a1b4911a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.049448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] RBAC: Authorizing `identity:delete_project(project_id=946496e8a67a4ced8c4efe81a1b4911a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.055519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.126272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946496e8a67a4ced8c4efe81a1b4911a', 'request_id': 'req-3a4a9b2d-5421-4312-9967-4471de43750e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:42.134327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946496e8a67a4ced8c4efe81a1b4911a', 'request_id': 'req-3a4a9b2d-5421-4312-9967-4471de43750e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:42.144711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946496e8a67a4ced8c4efe81a1b4911a', 'request_id': 'req-3a4a9b2d-5421-4312-9967-4471de43750e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:42.160004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] The token cache is being invalidate because project 946496e8a67a4ced8c4efe81a1b4911a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:42.160387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:42.161284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3a4a9b2d-5421-4312-9967-4471de43750e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:42.167309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 941/1883] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:41 2022] DELETE /identity/v3/projects/946496e8a67a4ced8c4efe81a1b4911a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:42.169200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.169902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.170199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.192256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:42.193560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:42.251762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:42.268910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 943/1884] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:42.282439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.282598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.283065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.335292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:42.337843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:42.351491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 942/1885] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:42.355831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.377584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed739eed-adfb-4bc9-974e-d1a4326b88b2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.389232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.390462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.390957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.391656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.394351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:42.394679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.398201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.415944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 944/1886] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:42.420824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.433457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373e654-29f2-4337-8a65-7e0baed555cc None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.446557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.447332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.447690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.448045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.448812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.451562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.489929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 943/1887] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:42 2022] POST /identity/v3/projects => generated 359 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:42.491577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.502455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2919-084b-4300-8e7b-d9d24a564ce9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.575216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.576051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.576426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.576772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.577640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.579634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.611529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 945/1888] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:42 2022] POST /identity/v3/users => generated 280 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:42.615795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.629013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566a874-23e5-4633-b639-7a9dff1e6747 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.640581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.641279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.641564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.641842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.642583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.644371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.657290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 944/1889] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:42.661531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.673071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61a2c1-2a6f-4b8a-afc9-0d3617cce051 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.678056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.678778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.679069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.679522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.680335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.682009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4448963-3845-42e3-b68f-1abae5517c58 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.707350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 946/1890] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:42.708563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.723376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef39ca5-ae43-4560-8e14-1f240ade0c4c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:42.733519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:42.734681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.735086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.735405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] PATH_INFO: `/v3/projects/5aedeff4583344c8ad28fda0af456d9e/users/108858cb0b624d84bb693a7fcae89d08/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.755368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5aedeff4583344c8ad28fda0af456d9e, user_id=108858cb0b624d84bb693a7fcae89d08, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:42.759000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:42.780968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 945/1891] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:42 2022] PUT /identity/v3/projects/5aedeff4583344c8ad28fda0af456d9e/users/108858cb0b624d84bb693a7fcae89d08/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:42.794281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:42.794965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:42.794965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:42.828305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] MFA Rules not processed for user `108858cb0b624d84bb693a7fcae89d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:42.829705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:42.922377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 947/1892] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:42 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:42.953622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:42.957681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.011222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:43.018063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dc4a8-f1f2-4e16-a805-ce91d97c27f5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.022140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:43.023054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.023340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.023621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.031968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:43.033434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a6187-3d51-4604-b0df-eb2a1cbe399d service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:43.042799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 946/1893] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:43.056788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:43.109128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:43.126053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee2392-c834-451d-80b7-5022f8cea0dc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.130727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:43.131602 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.132546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.132839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] PATH_INFO: `/v3/users/108858cb0b624d84bb693a7fcae89d08` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.134070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] RBAC: Authorizing `identity:delete_user(user_id=108858cb0b624d84bb693a7fcae89d08)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:43.136016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:43.197560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108858cb0b624d84bb693a7fcae89d08'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.206334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108858cb0b624d84bb693a7fcae89d08'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.207291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108858cb0b624d84bb693a7fcae89d08'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.222644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108858cb0b624d84bb693a7fcae89d08'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.232815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108858cb0b624d84bb693a7fcae89d08'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.239865 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 948/1894] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:43 2022] DELETE /identity/v3/users/108858cb0b624d84bb693a7fcae89d08 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:43.244355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.244768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.245151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.269778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:43.271159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.341343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 947/1895] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:43.351476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:43.353141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.424097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aae51c-3d0a-43fa-a22a-f591116a868e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.428373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:43.430436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.430731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.431020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.440601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:43.442051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:43.453014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 949/1896] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:43.619372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:43.620947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.633463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0aeb5-d82d-4a48-bbef-bd18079ad9a8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.638075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:43.638951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.639268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.639505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] PATH_INFO: `/v3/users/0820b46b177f497cac5f1bf0e9a73248` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.640728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0820b46b177f497cac5f1bf0e9a73248)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:43.642801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:43.692037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0820b46b177f497cac5f1bf0e9a73248'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.707846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0820b46b177f497cac5f1bf0e9a73248'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.708496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0820b46b177f497cac5f1bf0e9a73248'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.726473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0820b46b177f497cac5f1bf0e9a73248'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.727526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-239fd224-2028-461b-b2fd-40b2be254cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0820b46b177f497cac5f1bf0e9a73248'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.742674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:43.744714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.747178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 948/1897] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:43 2022] DELETE /identity/v3/users/0820b46b177f497cac5f1bf0e9a73248 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:43.748926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.749290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.749511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.768995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ab38c-79d6-4fbb-9377-c4192caec142 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.771213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:43.772673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.829219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:43.830090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.830370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.830645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] PATH_INFO: `/v3/projects/5aedeff4583344c8ad28fda0af456d9e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.831809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5aedeff4583344c8ad28fda0af456d9e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:43.833731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:43.835846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 949/1898] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:43.845651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:43.846457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:43.847088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:43.853634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:43.894430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aedeff4583344c8ad28fda0af456d9e', 'request_id': 'req-84c68212-8278-4442-b1a1-02e2fb35d411'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.901822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aedeff4583344c8ad28fda0af456d9e', 'request_id': 'req-84c68212-8278-4442-b1a1-02e2fb35d411'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.908115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aedeff4583344c8ad28fda0af456d9e', 'request_id': 'req-84c68212-8278-4442-b1a1-02e2fb35d411'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.920234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c91ff8-8aa0-4dd6-b50e-c4135d758a53 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:43.941685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] The token cache is being invalidate because project 5aedeff4583344c8ad28fda0af456d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:43.942658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.944941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84c68212-8278-4442-b1a1-02e2fb35d411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:43.947844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 950/1899] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:43 2022] DELETE /identity/v3/projects/5aedeff4583344c8ad28fda0af456d9e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:43.953767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd90c-95e4-4ec5-8f03-51b8855f291f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:43.954058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd90c-95e4-4ec5-8f03-51b8855f291f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:43.954337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dd90c-95e4-4ec5-8f03-51b8855f291f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:43.995766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ed9dd90c-95e4-4ec5-8f03-51b8855f291f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:43.998707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ed9dd90c-95e4-4ec5-8f03-51b8855f291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.027310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.028665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.029022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.029385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.040280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.111736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 951/1900] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:44.116279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.116869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.117646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.129533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.131030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.139533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:44.140920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.143213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 950/1901] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:44.150665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 952/1902] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:44.154913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.165708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482b4-d9a0-471a-bf1a-2c4efdab7fe0 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.170807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.171498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.171767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.172047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.172831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:44.173146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.174755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.193713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 951/1903] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:44.195505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.206523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b63eba-bac9-4a52-a93a-8c227db6c386 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.212481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.213171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.213453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.213750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.214442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.216691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.248633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 953/1904] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:44.250519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.266087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2239c7-d2e5-4464-aefb-0d61319ce33d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.347693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:44.357325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.358219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.358685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.358963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.359676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.361727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.384765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 952/1905] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:12:44.387844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.398857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501507d-c9e9-4df8-a9af-88e19c5ad97f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.404023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.404789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.405076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.405357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.406092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.407792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.423138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 954/1906] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:44.424325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.436229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b715b7-7245-493d-aa8b-06316b958cc3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.441459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.442206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.442567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.442920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.443657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.446547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79351712-bf08-4280-8a2b-55888388c11a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.457915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 953/1907] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:44.462651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.473058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cabe5-ac84-4db0-928c-569c1e9979b5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.478113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.478772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.479063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.479345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] PATH_INFO: `/v3/projects/9365180331be4a8f8736c8904f51e462/users/058f85aecda34fc8b04b04458184acb4/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.494770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9365180331be4a8f8736c8904f51e462, user_id=058f85aecda34fc8b04b04458184acb4, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.497654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.515237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 955/1908] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:44 2022] PUT /identity/v3/projects/9365180331be4a8f8736c8904f51e462/users/058f85aecda34fc8b04b04458184acb4/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:44.531026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.531026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.531026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.552421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.554211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.555749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] MFA Rules not processed for user `058f85aecda34fc8b04b04458184acb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:44.557038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.602246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:44.627046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28516749-ea09-4622-ba83-57d48ab0c131 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.630889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 954/1909] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:44.632372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.633503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.633791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.634071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] PATH_INFO: `/v3/projects/5c198a961c7c4e809578d2b156216b6d` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.635441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c198a961c7c4e809578d2b156216b6d)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.636007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.637392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.646202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c67db-cf2e-42d8-a0ab-960461103cd2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.651439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.652088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.652373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.652681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.653378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.655514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.683942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 955/1910] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:44.686079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c198a961c7c4e809578d2b156216b6d', 'request_id': 'req-497341f6-5613-4027-b505-739f81f60fd1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.688166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.692476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c198a961c7c4e809578d2b156216b6d', 'request_id': 'req-497341f6-5613-4027-b505-739f81f60fd1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.699502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c198a961c7c4e809578d2b156216b6d', 'request_id': 'req-497341f6-5613-4027-b505-739f81f60fd1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.723290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] The token cache is being invalidate because project 5c198a961c7c4e809578d2b156216b6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:44.723562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.724249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.726519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 956/1911] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:44 2022] DELETE /identity/v3/projects/5c198a961c7c4e809578d2b156216b6d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:44.730752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.732095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.762462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c17cc6-8d7a-4b52-a150-0f4f2e63a6cc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.767677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.768349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.768670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.768953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.769645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.771600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.792407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 956/1912] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:44 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:44.796230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.797822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:44.801702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497341f6-5613-4027-b505-739f81f60fd1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.807009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.807890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.808178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.808466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] PATH_INFO: `/v3/users/236069ada307444498e02d2b37fd2181` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.809686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236069ada307444498e02d2b37fd2181)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.811418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb69211-264b-40ac-9e65-1a15bcd867c3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.811627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.816042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.816721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.817008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.817287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.818009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.820394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.831194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 957/1913] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:44.835401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.846585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285a801-6c52-489c-b91a-5a4c289ad305 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:44.851700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:44.852602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:44.852980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:44.853261 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] PATH_INFO: `/v3/projects/9300b631eeab43ae937f0a9072910cbd/users/cff7890577b14a0c9273be8e58895503/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:44.861286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236069ada307444498e02d2b37fd2181'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.869071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236069ada307444498e02d2b37fd2181'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.869195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9300b631eeab43ae937f0a9072910cbd, user_id=cff7890577b14a0c9273be8e58895503, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:44.869748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236069ada307444498e02d2b37fd2181'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.871820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:44.874903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:44.881014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:44.885737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236069ada307444498e02d2b37fd2181'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.888611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 958/1914] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:44 2022] PUT /identity/v3/projects/9300b631eeab43ae937f0a9072910cbd/users/cff7890577b14a0c9273be8e58895503/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:44.892547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:44.898999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236069ada307444498e02d2b37fd2181'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:44.905320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 957/1915] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:44 2022] DELETE /identity/v3/users/236069ada307444498e02d2b37fd2181 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:44.958780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6c9e0-d572-4d7f-bd0d-8d924d3ffd95 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.028504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.029237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.029842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.029842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.030537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.032149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a44f9-7738-4c8a-a2d9-f6784ff850ef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:45.046769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 959/1916] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:45.047606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:45.049044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.063173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304320-6d99-4b55-91b4-2549917fac47 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.068497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.069581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.070198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.070615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] PATH_INFO: `/v3/projects/9300b631eeab43ae937f0a9072910cbd/users/cff7890577b14a0c9273be8e58895503/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.073066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9300b631eeab43ae937f0a9072910cbd, user_id=cff7890577b14a0c9273be8e58895503, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.076203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:45.092478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 958/1917] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:45 2022] PUT /identity/v3/projects/9300b631eeab43ae937f0a9072910cbd/users/cff7890577b14a0c9273be8e58895503/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:45.103298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.103624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.103862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.139546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] MFA Rules not processed for user `cff7890577b14a0c9273be8e58895503`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:45.140816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.226192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:45.227924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 960/1918] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:45.229035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.248851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:45.250276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.280427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:45.302080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330f582-4b91-4b57-b75f-8e163a2b2d1f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.307470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.308379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.308660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.309019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] PATH_INFO: `/v3/projects/394c06bc85e04d21b060b2abb5379594` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.310237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=394c06bc85e04d21b060b2abb5379594)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.312473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:45.313281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ce84f-1658-4390-90b6-02f1b3b0abe9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.318070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.318876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.320072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.320538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.324097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.367007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394c06bc85e04d21b060b2abb5379594', 'request_id': 'req-7a78c1ef-8176-4314-80a5-0e0be1c53e67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.378665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394c06bc85e04d21b060b2abb5379594', 'request_id': 'req-7a78c1ef-8176-4314-80a5-0e0be1c53e67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.381041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394c06bc85e04d21b060b2abb5379594', 'request_id': 'req-7a78c1ef-8176-4314-80a5-0e0be1c53e67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.398238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] The token cache is being invalidate because project 394c06bc85e04d21b060b2abb5379594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:45.398238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.398736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.400485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 959/1919] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:45 2022] DELETE /identity/v3/projects/394c06bc85e04d21b060b2abb5379594 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:45.404300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:45.405846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:45.407891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.409391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:45.419535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 961/1920] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:45.474661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78c1ef-8176-4314-80a5-0e0be1c53e67 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.481291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.482922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.483384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.483687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] PATH_INFO: `/v3/users/eb500bf7a7ba413bb9cf72ca3ea41db1` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.484971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb500bf7a7ba413bb9cf72ca3ea41db1)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.487378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:45.493100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:45.549790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb500bf7a7ba413bb9cf72ca3ea41db1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.558399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb500bf7a7ba413bb9cf72ca3ea41db1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.559159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb500bf7a7ba413bb9cf72ca3ea41db1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.576045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb500bf7a7ba413bb9cf72ca3ea41db1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.587407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e22b30fd-5d51-4382-a5b7-8345a63f7291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb500bf7a7ba413bb9cf72ca3ea41db1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:45.595814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 960/1921] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:45 2022] DELETE /identity/v3/users/eb500bf7a7ba413bb9cf72ca3ea41db1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:45.900957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:45.912118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201cb90-fe65-4306-89f9-968eafc59e7a service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:45.969786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:45.970642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:45.970928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:45.971206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] PATH_INFO: `/v3/projects/cf0f691f83694964b6c432670ca59fff` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:45.972379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0f691f83694964b6c432670ca59fff)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:45.974317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.021669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0f691f83694964b6c432670ca59fff', 'request_id': 'req-866b8b53-f265-4c83-a9f3-c8697076de75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:46.027729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0f691f83694964b6c432670ca59fff', 'request_id': 'req-866b8b53-f265-4c83-a9f3-c8697076de75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:46.034875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0f691f83694964b6c432670ca59fff', 'request_id': 'req-866b8b53-f265-4c83-a9f3-c8697076de75'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:46.045556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] The token cache is being invalidate because project cf0f691f83694964b6c432670ca59fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:46.045849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:46.046475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-866b8b53-f265-4c83-a9f3-c8697076de75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:46.048008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 962/1922] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:45 2022] DELETE /identity/v3/projects/cf0f691f83694964b6c432670ca59fff => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:46.054504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.054891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.055078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.073382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:46.074595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:46.137134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 961/1923] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:46.146692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.146988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.147271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.166407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:46.167542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:46.175620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 963/1924] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:46.180061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.189609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721106e6-c892-45c6-a489-35af6c7a81cd None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.194121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.195328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.195609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.195882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.196632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:46.196942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.198558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.211273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 962/1925] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:46.215586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.220292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:46.226711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516f27b-cf2a-489a-b50e-2f58a45eb8e3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.232625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.233282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.233570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.233959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.234662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.236622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.262119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 964/1926] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:46.266227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.277197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c41e46-e0bf-4340-8317-3f30aaf200b1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.336319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.337137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.337429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.337709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.338417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.340343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.356271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:46.365967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.368933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 963/1927] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:46.377854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20bad-245a-4974-bb34-f153f104d728 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.384836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.385522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.385809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.386204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.386843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.388431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.399831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 965/1928] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:46.403789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.414359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6149e30-561e-4abe-b297-d252d0c3982a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.419148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.419745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.420024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.420300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.421052 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.425016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9d932-94bd-44b4-aac2-e710ef923e11 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.441036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 964/1929] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:46.442432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.453259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af7341-9aaa-424c-bca3-9f128d66ba83 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.457960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.458841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.459122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.459401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] PATH_INFO: `/v3/projects/1e8c3c93b6b645a4b6fbc41cf936514a/users/61377e06f11340838b997c321c079a65/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.473946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8c3c93b6b645a4b6fbc41cf936514a, user_id=61377e06f11340838b997c321c079a65, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.476985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.489754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 966/1930] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:46 2022] PUT /identity/v3/projects/1e8c3c93b6b645a4b6fbc41cf936514a/users/61377e06f11340838b997c321c079a65/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:46.501963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.502255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.502536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.535182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] MFA Rules not processed for user `61377e06f11340838b997c321c079a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:46.536400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:46.604529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 965/1931] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:46.609099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.677165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986393f-4fdf-44f7-ae92-50c8b43d18fc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.682018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.682672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.682955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.683226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.683922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.685854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.711668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 967/1932] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:46.715945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.726877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430341b-6c60-4622-88fe-e78df20b989b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.796916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.797598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.797882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.798159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.798854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.800829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.830001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 966/1933] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:46 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:46.831038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.846413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498f998-42ee-4da2-a161-a5f7ae241deb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.852893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.853567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.853848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.854128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.854849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.856472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.868871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 968/1934] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:46.872846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.883415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc1aea-9f15-4110-afdb-db8cb9d6c0fd admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.888155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.889141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.889424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.889723 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] PATH_INFO: `/v3/projects/1c4d5b42e49744a588561ed618b88305/users/70649f8f6dbf40deaa90b7c6bfa115f0/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.905291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4d5b42e49744a588561ed618b88305, user_id=70649f8f6dbf40deaa90b7c6bfa115f0, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.907997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:46.921005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 967/1935] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:46 2022] PUT /identity/v3/projects/1c4d5b42e49744a588561ed618b88305/users/70649f8f6dbf40deaa90b7c6bfa115f0/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:46.925615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:46.936183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654f3ad-257c-4ba7-ac92-b1d44eb3b99e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:46.992273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:46.993122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:46.993416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:46.993677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:46.994392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:46.995995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c43799-e95f-4e5d-92c1-780d1332e3f7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:47.005869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 969/1936] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:12:47.010078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:47.020134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fff335-60b0-43e3-94eb-7e5af80e7545 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:47.024764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:47.025632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.025911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.026185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] PATH_INFO: `/v3/projects/1c4d5b42e49744a588561ed618b88305/users/70649f8f6dbf40deaa90b7c6bfa115f0/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.028111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4d5b42e49744a588561ed618b88305, user_id=70649f8f6dbf40deaa90b7c6bfa115f0, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:47.030783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:47.041707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 968/1937] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:47 2022] PUT /identity/v3/projects/1c4d5b42e49744a588561ed618b88305/users/70649f8f6dbf40deaa90b7c6bfa115f0/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:47.051670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.052116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.052244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.078673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] MFA Rules not processed for user `70649f8f6dbf40deaa90b7c6bfa115f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:47.079876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:47.083958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:47.153283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 970/1938] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:47.181194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:47.182919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:47.218543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:47.243081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f108f-cfee-49ec-b534-451637713b1f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:47.246709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:47.247359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.247647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.247929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.307257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:47.308730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ed19f-284e-41c9-bccd-30fb82708f4f service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:47.317639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 969/1939] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:56.560128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:56.561965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:56.659660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308a0d0-d0db-480d-80de-652ac5689ed4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:56.667613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:56.669336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:56.669681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:56.669969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] PATH_INFO: `/v3/users/058f85aecda34fc8b04b04458184acb4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:56.671837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=058f85aecda34fc8b04b04458184acb4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:56.675011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:56.737632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058f85aecda34fc8b04b04458184acb4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:56.749064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058f85aecda34fc8b04b04458184acb4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:56.751191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058f85aecda34fc8b04b04458184acb4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:56.766347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058f85aecda34fc8b04b04458184acb4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:56.767446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058f85aecda34fc8b04b04458184acb4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:56.790978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 971/1940] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:56 2022] DELETE /identity/v3/users/058f85aecda34fc8b04b04458184acb4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:56.792156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:56.796727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:56.797510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:56.822469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:56.833542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:56.931682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 970/1941] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:12:56.938549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:56.941654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:56.971374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:57.023733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2b5d1-ca7d-4924-a6d8-113ae2e1bd37 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:57.027900 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:57.028685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:57.029048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:57.029724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:57.041027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:57.043268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:57.064823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 972/1942] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:57.580786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:57.586117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:57.601124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216976a1-aff0-4784-8fdd-e3d9b4b1a9a0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:57.628646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:57.629540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:57.629826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:57.630107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] PATH_INFO: `/v3/projects/9365180331be4a8f8736c8904f51e462` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:57.633097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9365180331be4a8f8736c8904f51e462)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:57.638837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:57.776331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9365180331be4a8f8736c8904f51e462', 'request_id': 'req-053070ad-ce23-49a9-9c8e-bdcef256dec6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:57.785477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9365180331be4a8f8736c8904f51e462', 'request_id': 'req-053070ad-ce23-49a9-9c8e-bdcef256dec6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:57.799100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9365180331be4a8f8736c8904f51e462', 'request_id': 'req-053070ad-ce23-49a9-9c8e-bdcef256dec6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:57.824588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] The token cache is being invalidate because project 9365180331be4a8f8736c8904f51e462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:57.825014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:57.825848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:57.827945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 971/1943] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:57 2022] DELETE /identity/v3/projects/9365180331be4a8f8736c8904f51e462 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:57.833556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:57.858793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc225e-c498-406b-8718-181ae869f545 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:57.995153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:57.996338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:57.996828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:57.997297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] PATH_INFO: `/v3/users/cff7890577b14a0c9273be8e58895503` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:57.998954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff7890577b14a0c9273be8e58895503)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:58.001846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:58.071797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff7890577b14a0c9273be8e58895503'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.092945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff7890577b14a0c9273be8e58895503'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.094912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff7890577b14a0c9273be8e58895503'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.123628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff7890577b14a0c9273be8e58895503'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.125138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2728a711-6737-4950-8f1b-29dc7d7c82f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff7890577b14a0c9273be8e58895503'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.165981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 973/1944] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:12:57 2022] DELETE /identity/v3/users/cff7890577b14a0c9273be8e58895503 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:58.527822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:58.529274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:58.551313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053070ad-ce23-49a9-9c8e-bdcef256dec6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:58.561791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:58.643789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:58.644809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.645120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.645580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] PATH_INFO: `/v3/projects/9300b631eeab43ae937f0a9072910cbd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.647368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9300b631eeab43ae937f0a9072910cbd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:58.649912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:58.708718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9300b631eeab43ae937f0a9072910cbd', 'request_id': 'req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.717172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9300b631eeab43ae937f0a9072910cbd', 'request_id': 'req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.725539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9300b631eeab43ae937f0a9072910cbd', 'request_id': 'req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.754159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] The token cache is being invalidate because project 9300b631eeab43ae937f0a9072910cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:12:58.754460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.755113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c8ae68ac-70e1-4176-bb1b-8bd97d51170b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:12:58.761969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 972/1945] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:12:58 2022] DELETE /identity/v3/projects/9300b631eeab43ae937f0a9072910cbd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:12:58.762929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.762929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.762929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.811567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:58.812844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:58.860300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:58.898152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 974/1946] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:58.900014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.900548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.900630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.945022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:58.946579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:58.954861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 973/1947] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:58.959102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:58.971617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477858b4-ecd2-4a67-b1c2-9f69aafb67d5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:58.976891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:58.977840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.979108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.979624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.980889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:12:58.981439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:58.983310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.008732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 975/1948] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:59.012604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.027892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d690c-acaa-4429-aad1-e1b165bf1a67 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.033577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.034262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.034588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.034864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.035775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.038028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.066922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 974/1949] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:12:59 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:59.072019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.078422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.080318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:59.082516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e3102-3d99-465b-a636-d3020a993d80 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.141967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cb5a6-4d3b-4bc6-9e01-9b960a4c054a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.143853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.144643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.144999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.145350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.145817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.146196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.146447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.146732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.147394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.148519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.149377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:59.173301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 976/1950] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:12:59 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:12:59.177944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.189329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:59.189482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab0d79-25a5-4336-9d8f-1b40edb7e0aa admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.195369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.196104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.196603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.197060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.198160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.200251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.213213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 977/1951] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:59.217431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.228031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b41802-6630-4b56-89ed-82c82ce5569c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.236944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.238395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.241223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.241889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.242175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.242459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.243184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.244878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ede26-bd5c-4437-8fb8-e6ff7074ca3b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.248292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 975/1952] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:59.262080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 978/1953] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:12:59.267090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.278573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee13912-cb64-4a0c-bbb5-d968b55660bd service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.283877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.285029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.285313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.285585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] PATH_INFO: `/v3/projects/4011843ae6b14d4e9ed351847794b81c/users/a072c78f101249938d85d636d26473cd/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.304376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4011843ae6b14d4e9ed351847794b81c, user_id=a072c78f101249938d85d636d26473cd, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.308089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.329258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 976/1954] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:12:59 2022] PUT /identity/v3/projects/4011843ae6b14d4e9ed351847794b81c/users/a072c78f101249938d85d636d26473cd/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:12:59.340209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.340549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.341095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.370545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] MFA Rules not processed for user `a072c78f101249938d85d636d26473cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:59.371894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:59.455336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 979/1955] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:59.513912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:59.583218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f4047-809d-4d96-94a9-786289d54098 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:59.588384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:59.589317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:59.589614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:59.589896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:59.601716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:59.603603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bf9ec-b58e-430e-b3e8-5d457c28477b service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:59.624670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 977/1956] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:12:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:08.313108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:08.360342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:08.418396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90937db-d422-4c63-9ced-b1bfdd1ff04e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:08.423325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:08.424345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:08.425538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:08.426028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] PATH_INFO: `/v3/users/a072c78f101249938d85d636d26473cd` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:08.427351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a072c78f101249938d85d636d26473cd)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:08.430270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:08.501213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a072c78f101249938d85d636d26473cd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:08.511864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a072c78f101249938d85d636d26473cd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:08.512841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a072c78f101249938d85d636d26473cd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:08.539020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a072c78f101249938d85d636d26473cd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:08.539895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a072c78f101249938d85d636d26473cd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:08.579371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 980/1957] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:08 2022] DELETE /identity/v3/users/a072c78f101249938d85d636d26473cd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:08.589272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:08.589800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:08.590106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:08.622219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:08.624385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:08.719432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 978/1958] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:08.730822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:08.732088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:08.805566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f00d4-1fef-4258-8405-1170ee1edfb1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:08.809299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:08.809953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:08.810237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:08.810521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:08.821062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:08.822671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a9e1f-8ac6-47ff-ad18-54b33be44f69 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:08.832727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 981/1959] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:09.192658 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.194961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:09.212303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95704e6c-4ec2-45fb-89a4-76e92db4461b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.220346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.221545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.222243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.222687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] PATH_INFO: `/v3/projects/4011843ae6b14d4e9ed351847794b81c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.224087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4011843ae6b14d4e9ed351847794b81c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.227795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.286881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:09.308725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4011843ae6b14d4e9ed351847794b81c', 'request_id': 'req-909f3582-e954-4115-934d-c7086afe23c8'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:09.316056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4011843ae6b14d4e9ed351847794b81c', 'request_id': 'req-909f3582-e954-4115-934d-c7086afe23c8'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:09.325411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4011843ae6b14d4e9ed351847794b81c', 'request_id': 'req-909f3582-e954-4115-934d-c7086afe23c8'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:09.339603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] The token cache is being invalidate because project 4011843ae6b14d4e9ed351847794b81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:09.339897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:09.340567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-909f3582-e954-4115-934d-c7086afe23c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:09.342644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 979/1960] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:09 2022] DELETE /identity/v3/projects/4011843ae6b14d4e9ed351847794b81c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:09.347524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.347992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.348081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.367813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:09.369202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:09.429850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:09.449421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 982/1961] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:09.463522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591444-e54f-42e8-939a-834511aee813 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.463739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591444-e54f-42e8-939a-834511aee813 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.464091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591444-e54f-42e8-939a-834511aee813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.485410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b5591444-e54f-42e8-939a-834511aee813 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:09.486827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b5591444-e54f-42e8-939a-834511aee813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:09.495967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 980/1962] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:09.500237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.511221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dae03-d5b8-4621-8806-4c3cf7c090ce None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.516228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.516960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.517382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.517668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.518414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:09.518729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.520532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.534103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 983/1963] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:09.538886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591444-e54f-42e8-939a-834511aee813 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.548496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591444-e54f-42e8-939a-834511aee813 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.553838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.554493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.554772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.555044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.556298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.558311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.589599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 981/1964] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:09.590656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.601617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60faf3-da98-4253-ba9f-5f3bdeff3b31 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.670285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.670969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.671256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.671534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.672241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.674351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.697012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 984/1965] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:09.701023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.713146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a6c4d-8c59-4b09-a75c-583b3d291617 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.718584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.719335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.719694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.720046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.720889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.722595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.734176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 982/1966] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:09.738231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.749862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c663e9-585c-4e75-8189-3d4d2cc3ee84 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.755080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.755948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.757571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.757884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.758651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.760350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc1009-4c2f-44f9-b587-45dd11bd696c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.774229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 985/1967] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:09.775167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:09.785721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18734a05-8e6a-4578-a214-2997efbc45ee admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:09.791025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:09.791879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.792213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.792507 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] PATH_INFO: `/v3/projects/af4568548ed7494c98c140ad0ec813e6/users/263ee987fd88490e95e7857ebd17ee28/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.808203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4568548ed7494c98c140ad0ec813e6, user_id=263ee987fd88490e95e7857ebd17ee28, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:09.810993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:09.827715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 983/1968] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:09 2022] PUT /identity/v3/projects/af4568548ed7494c98c140ad0ec813e6/users/263ee987fd88490e95e7857ebd17ee28/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:09.838238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:09.838562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:09.838877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:09.869940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] MFA Rules not processed for user `263ee987fd88490e95e7857ebd17ee28`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:09.871175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:09.935178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 986/1969] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:09.939682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.004470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fb00a-573f-4030-a523-695e9329b6f0 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.009323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.010133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.010426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.010706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.011420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.013391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.040016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 984/1970] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:09 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:10.043849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.055464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976852-f7da-4a1c-9951-71cf1d61a50d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.099448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:10.119691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.120370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.120813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.121095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.121807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.123779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.132392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.135790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.144515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 987/1971] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:10.148623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.148969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1de3a5-0507-4852-8aa0-8b48a99ac944 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.153753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.154880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.155235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.155585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] PATH_INFO: `/v3/users/61377e06f11340838b997c321c079a65` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.156894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61377e06f11340838b997c321c079a65)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.158484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdf8dd-b7e6-406e-bcb2-57a97674e297 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.158970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.164271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.165209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.165478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.165753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.166468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.168059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.178787 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 988/1972] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:10.182917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.193802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52e4a4-5b0a-4e0f-b9ef-192ba6f03540 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.199344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.200223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.200801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.200955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.201743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.203570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:10.203879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.210662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61377e06f11340838b997c321c079a65'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.217163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 989/1973] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:10.218988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61377e06f11340838b997c321c079a65'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.219718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61377e06f11340838b997c321c079a65'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.221300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.231621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645d811-7ab9-404b-bbfa-9a6cdc7bdd2e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.232558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61377e06f11340838b997c321c079a65'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.236830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.237707 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.237994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.238274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] PATH_INFO: `/v3/projects/759d20c743a44b65b3287c8a5925efcd/users/a80e40aaa6784a5aabe77ca215716d79/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.241753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-faf733db-07b9-464f-8134-eb002eb85e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61377e06f11340838b997c321c079a65'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.248790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 985/1974] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:10 2022] DELETE /identity/v3/users/61377e06f11340838b997c321c079a65 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:10.253540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.253845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.254126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.254217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=759d20c743a44b65b3287c8a5925efcd, user_id=a80e40aaa6784a5aabe77ca215716d79, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.257168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfc8a0-e618-4206-97ad-36ee98fe33c0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.270783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 990/1975] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:10 2022] PUT /identity/v3/projects/759d20c743a44b65b3287c8a5925efcd/users/a80e40aaa6784a5aabe77ca215716d79/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:10.272738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:10.274034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.281203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.281497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.281784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.311412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] MFA Rules not processed for user `a80e40aaa6784a5aabe77ca215716d79`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:10.312771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.344596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 986/1976] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:10.355153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.357333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.391769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 991/1977] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:10.397613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.401535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.415875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b47a1b-808b-4f0c-aac3-3fed7dfcd30d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.420945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.421611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.421888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.422160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.422856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.425095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.433717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec6fc8-2049-4c41-a457-778a4bb4ab81 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.437763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.438482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.438857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.439202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.449143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.450586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.465304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 992/1978] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:10.466361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.467914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.469558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 987/1979] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:13:10.470728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.481579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04f7d8-ccbb-4c9b-8670-8954fc6ca36f service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.548062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:10.554338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d664233-bd2b-4af3-89f3-b0affa1b6316 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.561770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.562723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.563010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.563688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] PATH_INFO: `/v3/users/55bfc6f311d241e69e687efaebe1ea24` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.565484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bfc6f311d241e69e687efaebe1ea24)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.570154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.620294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.621284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.622106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.622776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.623742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.627752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.637024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:10.658438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bfc6f311d241e69e687efaebe1ea24'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.678442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 988/1980] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:10.695915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.703958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bfc6f311d241e69e687efaebe1ea24'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.706650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bfc6f311d241e69e687efaebe1ea24'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.729426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dac0f0-e973-4a66-9409-e0c92dd4096a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.736423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.737202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.737644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.737967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.738898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.740057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bfc6f311d241e69e687efaebe1ea24'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.741523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f4b31ae2-573d-472f-a0e6-c611c9346aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bfc6f311d241e69e687efaebe1ea24'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:10.741657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.757250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 989/1981] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:10.761027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 993/1982] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:10 2022] DELETE /identity/v3/users/55bfc6f311d241e69e687efaebe1ea24 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:10.764913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.768617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.769641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.770525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.772498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.797498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eb390-60cc-4b11-baab-81f7276316c7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:10.815130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:10.816386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.886166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:10.887345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:10.887634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:10.887917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] PATH_INFO: `/v3/projects/a7a4658ac020470bbb277afefd3f33e7/users/e4da293f6ac640ca9848c9b222524548/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:10.921229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 994/1983] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:10.926390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a4658ac020470bbb277afefd3f33e7, user_id=e4da293f6ac640ca9848c9b222524548, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:10.930952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:10.946099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:10.949745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:10.956430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 990/1984] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:10 2022] PUT /identity/v3/projects/a7a4658ac020470bbb277afefd3f33e7/users/e4da293f6ac640ca9848c9b222524548/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:10.963955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.056147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585185a7-20e2-421f-8fdc-133d167ce980 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.063008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9695e1-83ea-4266-95bb-e8e4a22247fa None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.076799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.077414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.077704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.077987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.080478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.083250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.128927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:11.135828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 991/1985] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:11.135828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.137966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.154634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b11de-1275-4022-bc4d-6bed5c40bc3b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.160212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.161115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.161406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.161706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] PATH_INFO: `/v3/projects/1e8c3c93b6b645a4b6fbc41cf936514a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.163936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8c3c93b6b645a4b6fbc41cf936514a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.165393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.166039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.166549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.166990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.167637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.183542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.185345 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.200643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 995/1986] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:11.202187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.219281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8346e0-0825-468e-af9c-d1a8f126e12c service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.232824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8c3c93b6b645a4b6fbc41cf936514a', 'request_id': 'req-f20c3f7d-2f93-48da-9f3a-32103726fd13'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.234731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:11.240078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8c3c93b6b645a4b6fbc41cf936514a', 'request_id': 'req-f20c3f7d-2f93-48da-9f3a-32103726fd13'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.249339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8c3c93b6b645a4b6fbc41cf936514a', 'request_id': 'req-f20c3f7d-2f93-48da-9f3a-32103726fd13'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.267265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] The token cache is being invalidate because project 1e8c3c93b6b645a4b6fbc41cf936514a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:11.267694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.268591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.270762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 992/1987] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:11 2022] DELETE /identity/v3/projects/1e8c3c93b6b645a4b6fbc41cf936514a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:11.271739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.274813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.323340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.324713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.325156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.325835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] PATH_INFO: `/v3/projects/a7a4658ac020470bbb277afefd3f33e7/users/e4da293f6ac640ca9848c9b222524548/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.328213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a4658ac020470bbb277afefd3f33e7, user_id=e4da293f6ac640ca9848c9b222524548, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.331642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.346484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 996/1988] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:11 2022] PUT /identity/v3/projects/a7a4658ac020470bbb277afefd3f33e7/users/e4da293f6ac640ca9848c9b222524548/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:11.347764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.349555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.397133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c3f7d-2f93-48da-9f3a-32103726fd13 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.457424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56959-a4bf-4e92-b238-8048af19f607 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.466039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.467179 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.467857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.468289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] PATH_INFO: `/v3/users/70649f8f6dbf40deaa90b7c6bfa115f0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.470921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70649f8f6dbf40deaa90b7c6bfa115f0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.476476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.478845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.479994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.480373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.480692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] PATH_INFO: `/v3/users/6bb159f8906243ee9a6ec6c7c3988a99` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.482484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb159f8906243ee9a6ec6c7c3988a99)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.485064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.542613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70649f8f6dbf40deaa90b7c6bfa115f0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.551706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70649f8f6dbf40deaa90b7c6bfa115f0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.553144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70649f8f6dbf40deaa90b7c6bfa115f0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.565819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb159f8906243ee9a6ec6c7c3988a99'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.572489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:11.574361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70649f8f6dbf40deaa90b7c6bfa115f0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.576273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3260183f-7098-4b55-ab98-5a93b9087cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70649f8f6dbf40deaa90b7c6bfa115f0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.576952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb159f8906243ee9a6ec6c7c3988a99'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.578455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb159f8906243ee9a6ec6c7c3988a99'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.597603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 997/1989] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:11 2022] DELETE /identity/v3/users/70649f8f6dbf40deaa90b7c6bfa115f0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:11.600275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.600512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb159f8906243ee9a6ec6c7c3988a99'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.600758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.601135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.611000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ec62fa7b-84e7-445e-85e2-6726003dbdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb159f8906243ee9a6ec6c7c3988a99'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.627348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 993/1990] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:11 2022] DELETE /identity/v3/users/6bb159f8906243ee9a6ec6c7c3988a99 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:11.634405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.635367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.635833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.643219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] MFA Rules not processed for user `e4da293f6ac640ca9848c9b222524548`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:11.644671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.658621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:11.668080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:11.672046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.752459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 994/1991] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:11.766409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.770435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.788008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 998/1992] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:11.789532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.790845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc6b97-a128-4967-8453-a25750f5a9ba None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.791152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.805087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.806160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.806329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.806615 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] PATH_INFO: `/v3/projects/7a4f2609a59542f9b6c6b314bf188f0e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.807980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4f2609a59542f9b6c6b314bf188f0e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.811378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.880169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed1d7-2a37-4f0c-9efa-8fd927d20e92 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:11.885012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:11.885693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:11.886170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:11.886463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:11.901023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:11.903262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:11.911400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4f2609a59542f9b6c6b314bf188f0e', 'request_id': 'req-f19f7973-a6b1-467e-ab55-d54a6be14312'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.918168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 999/1993] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:11.919113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:11.920693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:11.925470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4f2609a59542f9b6c6b314bf188f0e', 'request_id': 'req-f19f7973-a6b1-467e-ab55-d54a6be14312'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.937310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4f2609a59542f9b6c6b314bf188f0e', 'request_id': 'req-f19f7973-a6b1-467e-ab55-d54a6be14312'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.955914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] The token cache is being invalidate because project 7a4f2609a59542f9b6c6b314bf188f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:11.956256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.956983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:11.970808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 995/1994] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:11 2022] DELETE /identity/v3/projects/7a4f2609a59542f9b6c6b314bf188f0e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:12.051001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.052625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.056782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21280480-3881-49db-b099-30a7fbe35564 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.159573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:12.208470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.209884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.211083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.211399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.214119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.229380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f7973-a6b1-467e-ab55-d54a6be14312 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.235414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.236114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.236490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.236847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] PATH_INFO: `/v3/projects/1c4d5b42e49744a588561ed618b88305` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.238195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4d5b42e49744a588561ed618b88305)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.240892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.291111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:12.308130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.309871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790740-3e4f-4de4-ace1-244a65dc23d8 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.313624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4d5b42e49744a588561ed618b88305', 'request_id': 'req-2cf6251d-8143-4cc3-abe4-8edabd9b1013'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.324015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4d5b42e49744a588561ed618b88305', 'request_id': 'req-2cf6251d-8143-4cc3-abe4-8edabd9b1013'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.334814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1000/1995] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:12.344665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4d5b42e49744a588561ed618b88305', 'request_id': 'req-2cf6251d-8143-4cc3-abe4-8edabd9b1013'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.367627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] The token cache is being invalidate because project 1c4d5b42e49744a588561ed618b88305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:12.367939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.368688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.370999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 996/1996] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:12 2022] DELETE /identity/v3/projects/1c4d5b42e49744a588561ed618b88305 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:12.390501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594308-cc42-4910-846f-69ad9112060f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.392354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594308-cc42-4910-846f-69ad9112060f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.392775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05594308-cc42-4910-846f-69ad9112060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.426866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-05594308-cc42-4910-846f-69ad9112060f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:12.428954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-05594308-cc42-4910-846f-69ad9112060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.437496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.439190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.551806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1001/1997] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:12.555719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6251d-8143-4cc3-abe4-8edabd9b1013 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.561722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.561962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.562251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.562735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.563829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.564228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.564625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] PATH_INFO: `/v3/projects/b041a9f731074ff6bc97593c9dab81db` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.566388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b041a9f731074ff6bc97593c9dab81db)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.569788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.585663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:12.586951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.602362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1002/1998] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:12.603469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.618401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8e593-0ad7-42f6-b00d-7048a234887b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.625530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.626200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.626531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.626814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.627589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:12.627906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.629589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.638463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b041a9f731074ff6bc97593c9dab81db', 'request_id': 'req-bb4d6af4-5bdc-4235-a08c-ef821a45f894'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.653804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b041a9f731074ff6bc97593c9dab81db', 'request_id': 'req-bb4d6af4-5bdc-4235-a08c-ef821a45f894'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.663711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1003/1999] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:12.664947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.665080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b041a9f731074ff6bc97593c9dab81db', 'request_id': 'req-bb4d6af4-5bdc-4235-a08c-ef821a45f894'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.678625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6887cd2-3cc7-445a-9ef2-20eadb7b5e22 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.694856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] The token cache is being invalidate because project b041a9f731074ff6bc97593c9dab81db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:12.695296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.696779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.700028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 997/2000] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:12 2022] DELETE /identity/v3/projects/b041a9f731074ff6bc97593c9dab81db => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:12.704335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.706943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.772771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.773552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.774098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.774393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.776844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.779102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.786312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:12.794455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:12.806262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d6af4-5bdc-4235-a08c-ef821a45f894 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.814558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1004/2001] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:12 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:12.815648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.816698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.817044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.817414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] PATH_INFO: `/v3/users/46ed8fc893af4958b52bf588647c7a67` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.819042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.819514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ed8fc893af4958b52bf588647c7a67)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.821742 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.821965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:12.880158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ed8fc893af4958b52bf588647c7a67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.889717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ed8fc893af4958b52bf588647c7a67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.890463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ed8fc893af4958b52bf588647c7a67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.903647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ed8fc893af4958b52bf588647c7a67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.905113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6f65a-76e2-4811-af06-0d27e07ad5ae admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:12.911782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ed8fc893af4958b52bf588647c7a67'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:12.912806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:12.913588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:12.913941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:12.914462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:12.916106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:12.918523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:12.920051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 998/2002] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:12 2022] DELETE /identity/v3/users/46ed8fc893af4958b52bf588647c7a67 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:12.953913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1005/2003] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:12 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:12.957638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:12.959884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.102139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43537963-c17e-4a80-95e6-809f751b02cf admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.107450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.111676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.112820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.113207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.114089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.116916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.134169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 999/2004] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:13.138395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.152470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa4e32-1a66-47dc-99a3-7f9a558ad075 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.157798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.158487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.158773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.159584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.160187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.162916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065eb359-6db3-43cf-9775-9a7e0dddb592 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.177592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1006/2005] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:13.179789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.194323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab53d8-b160-421d-a7e7-a24234c2b864 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.202859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.204425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.204768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.206565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] PATH_INFO: `/v3/projects/09b8cd4a676c4ebab41612617d76321b/users/250b237759c44a688dfd4aa685ec4d8b/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.227315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b8cd4a676c4ebab41612617d76321b, user_id=250b237759c44a688dfd4aa685ec4d8b, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.230079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.264519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1000/2006] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:13 2022] PUT /identity/v3/projects/09b8cd4a676c4ebab41612617d76321b/users/250b237759c44a688dfd4aa685ec4d8b/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:13.277395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.278821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.278957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.280278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.281609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.345431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] MFA Rules not processed for user `250b237759c44a688dfd4aa685ec4d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:13.346816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.369670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:13.397847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07873a9d-9400-4c5d-92e9-e08655d1732c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.405744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.406892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.407209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.407493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] PATH_INFO: `/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.408799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6a8c05361440c6a24b12d3d1d08c15)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.411598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.442008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1007/2007] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:13.456976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.460266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.473223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6a8c05361440c6a24b12d3d1d08c15', 'request_id': 'req-9ba5b517-b063-457e-a158-5425270ab189'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:13.480494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6a8c05361440c6a24b12d3d1d08c15', 'request_id': 'req-9ba5b517-b063-457e-a158-5425270ab189'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:13.486690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6a8c05361440c6a24b12d3d1d08c15', 'request_id': 'req-9ba5b517-b063-457e-a158-5425270ab189'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:13.503298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] The token cache is being invalidate because project 3a6a8c05361440c6a24b12d3d1d08c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:13.503778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:13.505071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-9ba5b517-b063-457e-a158-5425270ab189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:13.507798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1001/2008] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:13 2022] DELETE /identity/v3/projects/3a6a8c05361440c6a24b12d3d1d08c15 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:13.514634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd38-92c6-4059-9c6a-576bcfa83a1c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.514634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd38-92c6-4059-9c6a-576bcfa83a1c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.514634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524cd38-92c6-4059-9c6a-576bcfa83a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.536730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d524cd38-92c6-4059-9c6a-576bcfa83a1c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:13.538027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d524cd38-92c6-4059-9c6a-576bcfa83a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.576662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db56b7-f319-46ce-9320-09650b0753bb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.597222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:13.627588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1002/2009] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:13.635917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.636273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.636673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.660050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:13.660050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.667392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.668317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.668693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.669088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.671124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:13.673368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1003/2010] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:13.674339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.687656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaf3f8-f7bd-4185-855e-a02f125ddc76 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.695477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.696353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.696761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.697114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.697924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:13.698303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.700358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.718115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1004/2011] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:13.719244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.738015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd518d6-ed70-4fd0-bef4-c7e7fdc9e0f8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.753004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.753709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.753998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.754278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.755115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.756876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.757471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.758963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:13.780967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1008/2012] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:13:13.790214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1005/2013] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:13 2022] POST /identity/v3/projects => generated 357 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:13.794495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:13.810667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e190fa2-512c-4501-b12b-7cca60371222 service glance] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:13.943599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:13.944827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:13.945802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:13.945802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:13.946824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:13.950031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.002375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.015827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1009/2014] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:13 2022] POST /identity/v3/users => generated 279 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:14.016291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831143-9660-4b2a-ab5e-33528f57b09c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.028428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.029201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.029491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.029773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.030571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.032555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.048600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1006/2015] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:14.052029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.063286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.065505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e550c-479a-405b-a0bd-4f69e8eb3c30 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.066044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:14.075965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.077139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.077567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.078086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.079378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.091965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.093760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:14.118258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1010/2016] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:14.122411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.139824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ba792-ed10-4d71-8625-3e1fea5a05a0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.142120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd11c55-1e37-4651-b065-20ce2fb3d3fc admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.145336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.146893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.147186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.147434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.147707 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] PATH_INFO: `/v3/projects/3f24fa07bf684021b9d68208de3afe78/users/af8b76959040409bbb6ce7c6d2543013/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.148438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.148900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.149479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.151622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:14.174676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f24fa07bf684021b9d68208de3afe78, user_id=af8b76959040409bbb6ce7c6d2543013, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.179547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d471ae0-5a6e-4d7e-b5b5-1c8b2ad15313 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.202389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1011/2017] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:14 2022] PUT /identity/v3/projects/3f24fa07bf684021b9d68208de3afe78/users/af8b76959040409bbb6ce7c6d2543013/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:14.214159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.214602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.214981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.253311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.255003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.255676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] MFA Rules not processed for user `af8b76959040409bbb6ce7c6d2543013`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:14.257705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:14.278315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:14.280318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1007/2018] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:13:14.442843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1012/2019] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:14.445019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.462451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad8bbb-f2b1-4368-828d-0593d2b1d7ca service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.582743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.583466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.584279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.584511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.585188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.587463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.631014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1008/2020] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:14 2022] POST /identity/v3/projects => generated 357 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:14.634941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.647990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537af6c-db92-4841-b31a-9f6c73ed07b4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.720587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.721327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.721601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.721845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.722663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.725167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.747153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1013/2021] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:14 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:14.751303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.762095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68a189-018c-449b-adac-ef2fccb1578c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.766693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.767354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.767634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.767911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.768660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.770383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.783118 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1009/2022] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:14.788033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.801410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e0a09-e1b0-4bcc-9263-462a4447d217 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.812853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.813818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.814106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.814389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] PATH_INFO: `/v3/projects/763da9d8ee17443a8400ab32b60cba54/users/7521767aa31349d5bd2c0f576096f449/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.843428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] RBAC: Authorizing `identity:create_grant(project_id=763da9d8ee17443a8400ab32b60cba54, user_id=7521767aa31349d5bd2c0f576096f449, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.846253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.873708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1014/2023] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:14 2022] PUT /identity/v3/projects/763da9d8ee17443a8400ab32b60cba54/users/7521767aa31349d5bd2c0f576096f449/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:14.874117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:14.885336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb733-6958-4b73-bf6a-e3d4910358f5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:14.940054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:14.972948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:14.973706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:14.974063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:14.974406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:14.976073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:14.978143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2451ec-1889-43e9-8d1a-03e109a26df7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:14.994959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1010/2024] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:14.996082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:15.017494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6139c2-dac9-4351-b3fe-c3758b98251f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:15.033133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:15.034561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:15.035010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:15.035442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] PATH_INFO: `/v3/projects/763da9d8ee17443a8400ab32b60cba54/users/7521767aa31349d5bd2c0f576096f449/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:15.038778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763da9d8ee17443a8400ab32b60cba54, user_id=7521767aa31349d5bd2c0f576096f449, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:15.042933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:15.063598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1015/2025] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:14 2022] PUT /identity/v3/projects/763da9d8ee17443a8400ab32b60cba54/users/7521767aa31349d5bd2c0f576096f449/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:15.075879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:15.076315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:15.076708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:15.117901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] MFA Rules not processed for user `7521767aa31349d5bd2c0f576096f449`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:15.119492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:15.180354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:15.246680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1011/2026] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:15.261574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:15.261953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682e555-70a4-4d96-9ff0-85c0997d1084 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:15.262978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:15.267351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:15.268316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:15.268811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:15.269110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:15.276519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:15.284133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc7b3-3789-4e62-8f3e-280e120e7fb0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:15.291665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:15.292578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:15.292893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:15.293262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:15.306353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:15.306686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:15.309367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:15.321567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1012/2027] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:15.368778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:15.370287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:15.395047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1016/2028] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:15.408631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:15.418664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475effce-f32e-4eb9-b1ab-71967da7ca5d service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:15.424494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:15.428500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:15.428500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:15.428500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:15.573964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:15.575447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:15.587147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1013/2029] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:17.732375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:17.734015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:17.802244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead85e0-d8c4-42fd-9ad1-2f7128ce32e4 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:17.807708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:17.808639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:17.808928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:17.809211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] PATH_INFO: `/v3/users/bf2f8a02c0ee46029b08a31d7ee9eec4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:17.810449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2f8a02c0ee46029b08a31d7ee9eec4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:17.812386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:17.859829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2f8a02c0ee46029b08a31d7ee9eec4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:17.867211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2f8a02c0ee46029b08a31d7ee9eec4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:17.867827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2f8a02c0ee46029b08a31d7ee9eec4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:17.881003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2f8a02c0ee46029b08a31d7ee9eec4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:17.881856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a3bb0cb-630c-4426-b4e8-a0cfa5765007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2f8a02c0ee46029b08a31d7ee9eec4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:17.895070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1017/2030] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:17 2022] DELETE /identity/v3/users/bf2f8a02c0ee46029b08a31d7ee9eec4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:18.106019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.123861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca012d51-6d14-4472-a81f-ca09152bce52 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.204279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:18.219011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.220247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.220608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.220888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] PATH_INFO: `/v3/projects/ef917e1df4a64838aba088201ddb8552` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.222145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef917e1df4a64838aba088201ddb8552)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.225082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.283373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef917e1df4a64838aba088201ddb8552', 'request_id': 'req-45971a44-665b-4d8f-a906-96ca063b2e03'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:18.293450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef917e1df4a64838aba088201ddb8552', 'request_id': 'req-45971a44-665b-4d8f-a906-96ca063b2e03'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:18.300096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef917e1df4a64838aba088201ddb8552', 'request_id': 'req-45971a44-665b-4d8f-a906-96ca063b2e03'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:18.323057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] The token cache is being invalidate because project ef917e1df4a64838aba088201ddb8552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:18.323395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:18.324335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-45971a44-665b-4d8f-a906-96ca063b2e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:18.326873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1014/2031] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:18 2022] DELETE /identity/v3/projects/ef917e1df4a64838aba088201ddb8552 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:18.332145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.332811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.333239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.358244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:18.359585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:18.400692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:18.440984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1018/2032] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:18.449572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.450084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.450243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.473753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:18.475522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:18.487336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1015/2033] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:18.492474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.504796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063526b-c4ab-4bd5-894e-160d0d7ff887 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.510274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.511038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.511390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.511678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.512413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:18.512768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.514476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.528513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1019/2034] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:18.533636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.543419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0e197-6eec-4ad4-bcb2-6125a19bfe07 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.548230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.548905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.549187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.549462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.550156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.552057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.577695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1016/2035] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:18 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:18.581950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.592941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cc67-9355-4605-8b52-4ec75142081e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.651319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.651988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.652278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.652590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.653299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.655251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.677186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1020/2036] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:18 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:18.682637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.695555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ff110-4f6c-40a4-b518-1e9d6c50f8ca admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.701550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.702427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.702714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.702991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.703759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.705427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.718791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1017/2037] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:18.722970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.734539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ab9b-870d-4533-a299-23977c267e90 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.740826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.741505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.741795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.742077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.742822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.744471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8568095-e74b-45ac-8f4e-1620e6211262 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.760968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1021/2038] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:18.761740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.774265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af4dde-2934-478a-be1f-ea4e1a94a19b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:18.779379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:18.781184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.781540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.782049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] PATH_INFO: `/v3/projects/706aa1db9c854954912661850ca7d9e3/users/d67e4e9573514ffc91815c90e72cf9fa/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.799835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=706aa1db9c854954912661850ca7d9e3, user_id=d67e4e9573514ffc91815c90e72cf9fa, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:18.803230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:18.817594 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1018/2039] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:18 2022] PUT /identity/v3/projects/706aa1db9c854954912661850ca7d9e3/users/d67e4e9573514ffc91815c90e72cf9fa/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:18.829453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:18.829453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:18.829453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:18.861929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] MFA Rules not processed for user `d67e4e9573514ffc91815c90e72cf9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:18.863144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:18.928534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1022/2040] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:18.945555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:18.947388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:19.020992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda18fb-eba5-4df8-b893-5cd7d7c6db4e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:19.027319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:19.027972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:19.028260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:19.028574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:19.038369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:19.040100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496a14f-1b3e-4e1a-8a07-e2c1cbc19791 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:19.050377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1019/2041] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:19.449477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:19.451270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:19.487676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:19.519672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a8aad-3cd7-4959-9540-af1bbf22c597 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:19.524997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:19.525878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:19.526165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:19.526467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] PATH_INFO: `/v3/users/263ee987fd88490e95e7857ebd17ee28` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:19.527667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=263ee987fd88490e95e7857ebd17ee28)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:19.529647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:19.579452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '263ee987fd88490e95e7857ebd17ee28'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:19.587162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '263ee987fd88490e95e7857ebd17ee28'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:19.587780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '263ee987fd88490e95e7857ebd17ee28'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:19.601952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '263ee987fd88490e95e7857ebd17ee28'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:19.602894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '263ee987fd88490e95e7857ebd17ee28'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:19.617488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1023/2042] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:19 2022] DELETE /identity/v3/users/263ee987fd88490e95e7857ebd17ee28 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:19.622454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:19.622923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:19.623076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:19.640940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:19.651570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:19.654272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:19.727903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1020/2043] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:19.737420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:19.738895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:19.803958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9538472-346c-47c3-9e2b-b0a6f4c45cc3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:19.808131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:19.808823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:19.809110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:19.809389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:19.819169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:19.820687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:19.830433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1024/2044] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:20.224170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:20.226966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:20.241108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db8af7-88ab-4384-a891-f399f0e262ec None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:20.253382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:20.254328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:20.254729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:20.255021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] PATH_INFO: `/v3/projects/af4568548ed7494c98c140ad0ec813e6` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:20.256264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4568548ed7494c98c140ad0ec813e6)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:20.258422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:20.306731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4568548ed7494c98c140ad0ec813e6', 'request_id': 'req-34424506-01f5-4631-8a87-5ece203ea41a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.312977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4568548ed7494c98c140ad0ec813e6', 'request_id': 'req-34424506-01f5-4631-8a87-5ece203ea41a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.319280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4568548ed7494c98c140ad0ec813e6', 'request_id': 'req-34424506-01f5-4631-8a87-5ece203ea41a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.331465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] The token cache is being invalidate because project af4568548ed7494c98c140ad0ec813e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:20.331751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.332377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.334807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1021/2045] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:20 2022] DELETE /identity/v3/projects/af4568548ed7494c98c140ad0ec813e6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:20.339681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:20.350279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0721-5f5f-4817-918e-6ec8a91a6192 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:20.402458 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:20.413652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:20.414532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:20.414817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:20.415105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] PATH_INFO: `/v3/users/a80e40aaa6784a5aabe77ca215716d79` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:20.416272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80e40aaa6784a5aabe77ca215716d79)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:20.418297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:20.468832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80e40aaa6784a5aabe77ca215716d79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.478017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80e40aaa6784a5aabe77ca215716d79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.478686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80e40aaa6784a5aabe77ca215716d79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.492808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80e40aaa6784a5aabe77ca215716d79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.493518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80e40aaa6784a5aabe77ca215716d79'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.505949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1025/2046] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:20 2022] DELETE /identity/v3/users/a80e40aaa6784a5aabe77ca215716d79 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:20.808531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:20.810096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:20.826459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34424506-01f5-4631-8a87-5ece203ea41a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:20.892765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:20.893622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:20.893909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:20.894189 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] PATH_INFO: `/v3/projects/759d20c743a44b65b3287c8a5925efcd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:20.895363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=759d20c743a44b65b3287c8a5925efcd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:20.897429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:20.931628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:20.946337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759d20c743a44b65b3287c8a5925efcd', 'request_id': 'req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.953260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759d20c743a44b65b3287c8a5925efcd', 'request_id': 'req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.959143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759d20c743a44b65b3287c8a5925efcd', 'request_id': 'req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.972023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] The token cache is being invalidate because project 759d20c743a44b65b3287c8a5925efcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:20.972310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.972952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:20.974649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1022/2047] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:20 2022] DELETE /identity/v3/projects/759d20c743a44b65b3287c8a5925efcd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:20.979076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:20.980546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:20.994497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a097d6d-1f8d-436d-87fb-c250f6781775 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.051327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.052121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.052936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.053365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] PATH_INFO: `/v3/users/e4da293f6ac640ca9848c9b222524548` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.054738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4da293f6ac640ca9848c9b222524548)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.056701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.104394 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4da293f6ac640ca9848c9b222524548'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.117810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4da293f6ac640ca9848c9b222524548'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.118611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4da293f6ac640ca9848c9b222524548'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.148008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4da293f6ac640ca9848c9b222524548'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.148849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-25f72636-f139-4dd5-9bcb-bfefeaf671f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4da293f6ac640ca9848c9b222524548'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.151812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:21.177370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1026/2048] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:20 2022] DELETE /identity/v3/users/e4da293f6ac640ca9848c9b222524548 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:21.461555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.463027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:21.476934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c471db5-c0f6-4986-bb7c-8d5cb1cecda6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.536724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.538078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.538358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.538634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] PATH_INFO: `/v3/projects/a7a4658ac020470bbb277afefd3f33e7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.539818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a4658ac020470bbb277afefd3f33e7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.541775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.588780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a4658ac020470bbb277afefd3f33e7', 'request_id': 'req-666cf73a-4189-4938-b1e3-12f4833249d6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.594900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a4658ac020470bbb277afefd3f33e7', 'request_id': 'req-666cf73a-4189-4938-b1e3-12f4833249d6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.601183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a4658ac020470bbb277afefd3f33e7', 'request_id': 'req-666cf73a-4189-4938-b1e3-12f4833249d6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.613352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] The token cache is being invalidate because project a7a4658ac020470bbb277afefd3f33e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:21.613637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.614255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-666cf73a-4189-4938-b1e3-12f4833249d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:21.616113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1023/2049] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:21 2022] DELETE /identity/v3/projects/a7a4658ac020470bbb277afefd3f33e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:21.621156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56158793-0d17-434c-9929-2094578b81fd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.621449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56158793-0d17-434c-9929-2094578b81fd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.621732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56158793-0d17-434c-9929-2094578b81fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.639817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-56158793-0d17-434c-9929-2094578b81fd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:21.641046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-56158793-0d17-434c-9929-2094578b81fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:21.701467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1027/2050] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:21.709966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.710258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.710538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.729682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:21.730873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:21.738778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1024/2051] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:21.743219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56158793-0d17-434c-9929-2094578b81fd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.752924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56158793-0d17-434c-9929-2094578b81fd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.757661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.758318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.758606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.758887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.759617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:21.759929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.761599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.773862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1028/2052] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:21.778023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.787770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895badc-b64f-4710-93ae-b05181e6299c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.792479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.793135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.793444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.793726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.794418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.796320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.823949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1025/2053] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:21 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:21.827915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.839265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7eda8-b08f-4596-b5cd-d45fda6b2d50 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.906374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.907116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.907424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.907728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.908517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.910609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.931321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1029/2054] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:21 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:21.935922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.947396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6503a-aeb9-43ee-a80b-de04074eb29a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.952336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.953072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.953361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.953665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.954547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.956606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:21.968336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1026/2055] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:21.972307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:21.983456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2347c-8c08-42fa-bc4d-fe27d600c06d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:21.988711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:21.989384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:21.989700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:21.990567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:21.991486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:21.993258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596dd64-41f2-4379-a412-804582ecc54d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:22.003798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1030/2056] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:22.008090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:22.018141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd876-21f4-4c3f-9b68-fc051c4c3aa7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:22.022919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:22.023797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:22.024212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:22.024352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] PATH_INFO: `/v3/projects/b0cef4ad08c545d982455e92acdeb258/users/f7691dd3d55048319211983b5aa8973d/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:22.032402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:22.042372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0cef4ad08c545d982455e92acdeb258, user_id=f7691dd3d55048319211983b5aa8973d, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:22.045145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:22.059186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1027/2057] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:22 2022] PUT /identity/v3/projects/b0cef4ad08c545d982455e92acdeb258/users/f7691dd3d55048319211983b5aa8973d/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:22.070450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:22.070795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:22.071149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:22.098808 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] MFA Rules not processed for user `f7691dd3d55048319211983b5aa8973d`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:22.100596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:22.146332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:22.167698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1031/2058] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:22.179503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:22.181324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:22.239449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66e55-9823-431d-9d11-869d60be85a2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:22.243875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:22.244624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:22.244905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:22.245180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:22.253471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:22.254894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3608-59f0-41ae-80de-6e0808230e40 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:22.263999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1028/2059] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:25.997339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:26.071415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b5314-b2fa-4af9-a6c8-54bba91abf10 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:26.075938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:26.076843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.077122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.077403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] PATH_INFO: `/v3/users/f7691dd3d55048319211983b5aa8973d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.078588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7691dd3d55048319211983b5aa8973d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:26.080597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:26.146480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7691dd3d55048319211983b5aa8973d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.155975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7691dd3d55048319211983b5aa8973d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.156739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7691dd3d55048319211983b5aa8973d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.171057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7691dd3d55048319211983b5aa8973d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.171756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7691dd3d55048319211983b5aa8973d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.187213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1032/2060] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:25 2022] DELETE /identity/v3/users/f7691dd3d55048319211983b5aa8973d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:26.192502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.193794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.193794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.213502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:26.215021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:26.292299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1029/2061] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:26.301862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:26.304337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:26.372720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff246bb-f724-4a76-ba42-b78f0ce670d6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:26.376543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:26.377220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.377509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.377786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.389299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:26.390437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb5878-e05b-4015-b028-1fe52cb5720e service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:26.401316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1033/2062] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:26.718001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:26.719513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:26.740947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb054c8-f0a7-4311-a338-b4706ca48620 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:26.751111 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:26.752035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.752397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.752746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] PATH_INFO: `/v3/projects/b0cef4ad08c545d982455e92acdeb258` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.753930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0cef4ad08c545d982455e92acdeb258)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:26.756697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:26.804465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0cef4ad08c545d982455e92acdeb258', 'request_id': 'req-5a32914b-1c8f-4cc4-9356-fed597a071e9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.810686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0cef4ad08c545d982455e92acdeb258', 'request_id': 'req-5a32914b-1c8f-4cc4-9356-fed597a071e9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.816248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0cef4ad08c545d982455e92acdeb258', 'request_id': 'req-5a32914b-1c8f-4cc4-9356-fed597a071e9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.829146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] The token cache is being invalidate because project b0cef4ad08c545d982455e92acdeb258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:26.829433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.830026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a32914b-1c8f-4cc4-9356-fed597a071e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:26.831638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1030/2063] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:26 2022] DELETE /identity/v3/projects/b0cef4ad08c545d982455e92acdeb258 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:26.836714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.837005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.837287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.855743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:26.856966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:26.861064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:26.917322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1034/2064] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:26.925217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64653595-f720-469d-8163-a83437cc967d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.925502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64653595-f720-469d-8163-a83437cc967d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.925779 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64653595-f720-469d-8163-a83437cc967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.944115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-64653595-f720-469d-8163-a83437cc967d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:26.945376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-64653595-f720-469d-8163-a83437cc967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:26.953373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1031/2065] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:26.958104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:26.970049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890726eb-9371-4232-b687-e58e52e80a14 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:26.975864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:26.976623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:26.976905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:26.977190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:26.977910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:26.978216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:26.979831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:26.992288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1035/2066] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:13:26.996836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64653595-f720-469d-8163-a83437cc967d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.006361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64653595-f720-469d-8163-a83437cc967d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.011187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.011838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.012124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.012405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.013209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.015125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.018373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:27.047560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1032/2067] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:26 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:27.051840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.062343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477f21-e115-4ce3-b91e-f3d8ea158c1c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.118881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.119771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.120052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.120739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.121033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.122966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.143699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1036/2068] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:27 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:27.148158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.159243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d373f-82f2-4087-b3b4-5cb75e68536c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.164878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.165548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.165832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.166228 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.166968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.168623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.179936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1033/2069] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:27.183986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.195001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c713427-fc1b-450f-bad2-a5df3258b55d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.199907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.200784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.201073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.201352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.202272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.204046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c26862-20f5-4c76-aa73-4120332a41e9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.215338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1037/2070] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:27.220577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.230253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba16e4-3de5-4a67-87df-f53cb36dbbf7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.235318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.236216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.236528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.236809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] PATH_INFO: `/v3/projects/9b5a8f5cf5694164bd6bfa640178de4b/users/d3472b6d287a493b9239175512b6d9cc/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.253253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5a8f5cf5694164bd6bfa640178de4b, user_id=d3472b6d287a493b9239175512b6d9cc, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.259133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.275563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1034/2071] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:27 2022] PUT /identity/v3/projects/9b5a8f5cf5694164bd6bfa640178de4b/users/d3472b6d287a493b9239175512b6d9cc/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:27.290109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.291826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.291826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.340663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] MFA Rules not processed for user `d3472b6d287a493b9239175512b6d9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:27.343167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:27.441853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1038/2072] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:27.458092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.461476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:27.477924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.479873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:27.487770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:27.572979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95468f4e-d24e-4d4a-8dd5-a743472c5350 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.578979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.580053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.580564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.580962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038fad4-cb9e-4389-8e55-95867500b101 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.581092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.586305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.587618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.587901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.588178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] PATH_INFO: `/v3/users/d67e4e9573514ffc91815c90e72cf9fa` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.590217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67e4e9573514ffc91815c90e72cf9fa)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.591894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.592481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.593939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397e1a7-5322-4d7e-9573-e1ed061846a9 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.604617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1035/2073] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:13:27.642136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67e4e9573514ffc91815c90e72cf9fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:27.650268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67e4e9573514ffc91815c90e72cf9fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:27.651136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67e4e9573514ffc91815c90e72cf9fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:27.665380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67e4e9573514ffc91815c90e72cf9fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:27.666106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67e4e9573514ffc91815c90e72cf9fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:27.682052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1039/2074] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:27 2022] DELETE /identity/v3/users/d67e4e9573514ffc91815c90e72cf9fa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:27.686697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.687088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.687439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.705575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:27.706791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:27.796944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1036/2075] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:27.810284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:27.810455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:27.887367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935d3c-dab8-421e-b53c-d91f96429ddf admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:27.895408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:27.897966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:27.900657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:27.901459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:27.922171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:27.924646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2daec4-6c7f-47e4-83f2-973aee2cf1c6 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:27.946071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1040/2076] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:28.247744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.249257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:28.264329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e868d-e8df-41dc-8249-b2735fe72580 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.269786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.270677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.271768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.271768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] PATH_INFO: `/v3/projects/706aa1db9c854954912661850ca7d9e3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.273628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=706aa1db9c854954912661850ca7d9e3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.276098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.335477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:28.336893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '706aa1db9c854954912661850ca7d9e3', 'request_id': 'req-8935e706-78dc-4c5d-aa79-61787c7b1965'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:28.344383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '706aa1db9c854954912661850ca7d9e3', 'request_id': 'req-8935e706-78dc-4c5d-aa79-61787c7b1965'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:28.350744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '706aa1db9c854954912661850ca7d9e3', 'request_id': 'req-8935e706-78dc-4c5d-aa79-61787c7b1965'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:28.363521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] The token cache is being invalidate because project 706aa1db9c854954912661850ca7d9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:28.363815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:28.364454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8935e706-78dc-4c5d-aa79-61787c7b1965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:28.376596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1037/2077] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:28 2022] DELETE /identity/v3/projects/706aa1db9c854954912661850ca7d9e3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:28.383220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.384175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.384662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.407654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:28.409022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:28.439453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:28.493196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1041/2078] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:28.500988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.501277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.501557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.520622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:28.521927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:28.530056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1038/2079] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:28.534273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.545022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98d042-0252-4c59-8034-496ccd1e1774 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.551272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.551940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.552843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.553307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.554051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:28.554365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.556039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.569884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1042/2080] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:28.574508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.584318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e942e-7137-4091-98a4-ff110d8cbd92 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.590004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.591078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.591379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.591656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.592358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.594355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.620428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1039/2081] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:28 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:28.624713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.636378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6786b50b-9eb2-413e-b798-73f6703c4a07 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.705470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.706148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.706432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.706711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.707415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.710040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.732884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1043/2082] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:28 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:28.737121 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.748961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ef93-3bc0-4300-a073-e4bae58e3540 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.753847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.756225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.756642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.757044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.757918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.761256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.775265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1040/2083] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:13:28.779550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.791152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039eeca-364c-42af-9108-0ce8399e4e58 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.798152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.798836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.799104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.799380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.800104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.801815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8c723-f584-47cc-b2c1-137318080bc2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.814045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1044/2084] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:28.818763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:28.828977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9ee3d-1a99-4f28-88b0-a74ba30d34f4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:28.834292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:28.835259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.835542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.835819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] PATH_INFO: `/v3/projects/3d91fef56e7040d5a8c75b39fb8b1c34/users/e1b7e8915bb941b38398497f9a1d77ec/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.853493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d91fef56e7040d5a8c75b39fb8b1c34, user_id=e1b7e8915bb941b38398497f9a1d77ec, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:28.856224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:28.873227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1041/2085] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:28 2022] PUT /identity/v3/projects/3d91fef56e7040d5a8c75b39fb8b1c34/users/e1b7e8915bb941b38398497f9a1d77ec/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:28.883492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:28.883807 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:28.884090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:28.914560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] MFA Rules not processed for user `e1b7e8915bb941b38398497f9a1d77ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:28.915777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:29.013084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1045/2086] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:29.018581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.097647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006d651-858f-4516-8732-a0aa501565c9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.103285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.105202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.105202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.105202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.105202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.108048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.140438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1042/2087] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:29 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:29.144783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.158744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce41e3-ece5-4e13-a1ff-473c118a280a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.198214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:29.254802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.255651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.255987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.256371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.257167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.259228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.288069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1046/2088] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:29 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:29.293308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.306579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7804fee-d017-4373-bd27-08e8a522bc04 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.313407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.314322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.314671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.315198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.316113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.317858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.332500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1043/2089] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:29.335325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.347338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2bda6-42a6-495f-9154-9ce643297a75 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.353622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.354592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.354885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.355157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] PATH_INFO: `/v3/projects/dd0870af27dd4bdfb7348214ab636c18/users/b56d43ede8da45c882d0e790b3e3eef6/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.375256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0870af27dd4bdfb7348214ab636c18, user_id=b56d43ede8da45c882d0e790b3e3eef6, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.378095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.394503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1047/2090] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:29 2022] PUT /identity/v3/projects/dd0870af27dd4bdfb7348214ab636c18/users/b56d43ede8da45c882d0e790b3e3eef6/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:29.401036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.412347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d713c3f-7a21-48e5-b910-31721fea0b42 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.465056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:29.481104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.481782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.482068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.482349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.483088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.485128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9229-5ece-4e6a-929c-35d51e4a85dc admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.497028 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1044/2091] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:29.502155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.515236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b5c7d-ca9c-429f-a4bc-e0dec95aa3b4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.520471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.522150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.522450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.522746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] PATH_INFO: `/v3/projects/dd0870af27dd4bdfb7348214ab636c18/users/b56d43ede8da45c882d0e790b3e3eef6/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.525212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0870af27dd4bdfb7348214ab636c18, user_id=b56d43ede8da45c882d0e790b3e3eef6, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.528149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.542505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1048/2092] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:29 2022] PUT /identity/v3/projects/dd0870af27dd4bdfb7348214ab636c18/users/b56d43ede8da45c882d0e790b3e3eef6/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:29.556000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.556439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.557049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.594537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] MFA Rules not processed for user `b56d43ede8da45c882d0e790b3e3eef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:29.595789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:29.679459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1045/2093] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:29.689029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.691205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:29.762934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b61ac-915a-4417-a379-4e38af633510 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.767542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.768190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.768587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.768868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.778870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.780460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.799341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1049/2094] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:29.859937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:29.862048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:29.878966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670bbdd-70f5-45cc-9736-e1b4cc1e9a1a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:29.882924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:29.883611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:29.883898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:29.884248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:29.960754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:29.962136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204238fd-a731-4b1e-a218-818aec3e3828 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:29.973380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1046/2095] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:34.249280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:34.251206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:34.328336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf953d-157f-4b78-880a-e3f85c41f6f5 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:34.332921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:34.334333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:34.334611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:34.334883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] PATH_INFO: `/v3/users/af8b76959040409bbb6ce7c6d2543013` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:34.336191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8b76959040409bbb6ce7c6d2543013)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:34.338210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:34.361722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:34.388866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8b76959040409bbb6ce7c6d2543013'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.397410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8b76959040409bbb6ce7c6d2543013'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.398044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8b76959040409bbb6ce7c6d2543013'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.409708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8b76959040409bbb6ce7c6d2543013'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.410341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8b76959040409bbb6ce7c6d2543013'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.422865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1050/2096] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:34 2022] DELETE /identity/v3/users/af8b76959040409bbb6ce7c6d2543013 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:34.428015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:34.428520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:34.428613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:34.445826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:34.447017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:34.505084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1047/2097] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:34.515133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:34.516567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:34.573090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b350a70-67b8-4c49-b094-bcbbbd9bf7b0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:34.576944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:34.577597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:34.577883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:34.578163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:34.587480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:34.588967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:34.598581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1051/2098] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:34.868238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:34.869681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:34.881706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965fc46-dfb3-45f7-bff6-0e404dd81b85 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:34.886577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:34.887522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:34.887804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:34.888081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] PATH_INFO: `/v3/projects/3f24fa07bf684021b9d68208de3afe78` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:34.889284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f24fa07bf684021b9d68208de3afe78)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:34.891256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:34.893022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:34.938700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f24fa07bf684021b9d68208de3afe78', 'request_id': 'req-bab2b679-464f-4fd7-b618-838dd5c99dfd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.944681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f24fa07bf684021b9d68208de3afe78', 'request_id': 'req-bab2b679-464f-4fd7-b618-838dd5c99dfd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.950193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f24fa07bf684021b9d68208de3afe78', 'request_id': 'req-bab2b679-464f-4fd7-b618-838dd5c99dfd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.964144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] The token cache is being invalidate because project 3f24fa07bf684021b9d68208de3afe78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:34.964555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.965302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:34.967212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1048/2099] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:34 2022] DELETE /identity/v3/projects/3f24fa07bf684021b9d68208de3afe78 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:34.971326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:34.982197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d93ad-f9ba-4819-acb9-f85c5d432c1c service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.038635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.039424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.039700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.040000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] PATH_INFO: `/v3/users/7521767aa31349d5bd2c0f576096f449` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.041190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7521767aa31349d5bd2c0f576096f449)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.043118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.082918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:35.088973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7521767aa31349d5bd2c0f576096f449'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.096736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7521767aa31349d5bd2c0f576096f449'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.097337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7521767aa31349d5bd2c0f576096f449'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.110535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7521767aa31349d5bd2c0f576096f449'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.111439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5c79e0d2-bdf0-40e4-83dc-985f08364d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7521767aa31349d5bd2c0f576096f449'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.125308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1052/2100] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:34 2022] DELETE /identity/v3/users/7521767aa31349d5bd2c0f576096f449 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:35.397088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.398705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:35.414130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2b679-464f-4fd7-b618-838dd5c99dfd admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.477537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.478403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.478684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.478960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] PATH_INFO: `/v3/projects/763da9d8ee17443a8400ab32b60cba54` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.480146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=763da9d8ee17443a8400ab32b60cba54)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.482206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.530054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763da9d8ee17443a8400ab32b60cba54', 'request_id': 'req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.536296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763da9d8ee17443a8400ab32b60cba54', 'request_id': 'req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.542924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763da9d8ee17443a8400ab32b60cba54', 'request_id': 'req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.556714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] The token cache is being invalidate because project 763da9d8ee17443a8400ab32b60cba54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:35.556986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.557604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ef5cefba-e2d8-402f-ae8e-16eba1dc1e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:35.559163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1049/2101] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:35 2022] DELETE /identity/v3/projects/763da9d8ee17443a8400ab32b60cba54 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:35.564023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.564504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.564727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.582870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:35.584302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:35.642431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1053/2102] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:35.649991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.650283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.650563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.670003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:35.671196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:35.680322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1050/2103] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:35.684528 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.694568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c104ba4-a3de-4a21-b851-85d3fe7d11e5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.699219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.699871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.700159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.700440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.701197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:35.701510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.703165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.715047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1054/2104] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:35.719691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.729853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0b5fc-2b8c-4ebc-afca-b406bf133f96 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.734591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.735238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.735524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.735803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.736556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.738472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.749105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:35.770478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1051/2105] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:35 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:35.771605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.781632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c3e1-d386-4359-8c09-403d899da6ea admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.839828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.840524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.841069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.841347 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.842028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.843957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.863631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1055/2106] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:35 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:35.867744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.877828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbc3b-7d2b-4f86-9444-a0e232b78f49 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.883012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.883674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.883955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.884232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.885008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.886647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.897505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1052/2107] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:35.901404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.911328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1902545-df1c-4209-9ce9-bf3710ee3f20 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.916657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.917333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.917622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.917901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.918618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.920231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dbde3-7e32-4b6c-9d2c-e71a9adfccc9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.930272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1056/2108] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:35.934294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:35.944400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b0d6b-55fa-447f-93ed-2a91f6395a6b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:35.948954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:35.949813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.950094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.950366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] PATH_INFO: `/v3/projects/c88d1ec683574cadabfa671988b58f90/users/1a8d7ced0251440699c0d9ffe413459a/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:35.965285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88d1ec683574cadabfa671988b58f90, user_id=1a8d7ced0251440699c0d9ffe413459a, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:35.968255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:35.981652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1053/2109] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:35 2022] PUT /identity/v3/projects/c88d1ec683574cadabfa671988b58f90/users/1a8d7ced0251440699c0d9ffe413459a/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:35.993445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:35.993759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:35.994018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.019961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] MFA Rules not processed for user `1a8d7ced0251440699c0d9ffe413459a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:36.021737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:36.033553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:36.088229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1057/2110] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:36.092993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.155492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893f5ad-196f-4446-9bf6-7cb99036704b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.160059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.160759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.161042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.161316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.162988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.164942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.189994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1054/2111] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:36 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:36.194766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.206690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4a205-b517-4d06-8c64-eb41201f77d9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.263091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.263750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.264032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.264306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.265021 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.266949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.287323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1058/2112] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:36 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:36.291496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.301788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb198-99a0-4949-8ad8-a956f2065b17 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.306618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.307303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.307582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.307857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.308613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.310244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.321591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1055/2113] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:36.325611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.336281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf8a4f-3499-4dbd-bb6d-4dc5fe6c28c4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.341077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.341943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.342221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.342495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] PATH_INFO: `/v3/projects/fee5ffe33cd94866a090a4ea94938ce5/users/2da300340e074443a74e142d688b9ad5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.358224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee5ffe33cd94866a090a4ea94938ce5, user_id=2da300340e074443a74e142d688b9ad5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.361124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.378423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1059/2114] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:36 2022] PUT /identity/v3/projects/fee5ffe33cd94866a090a4ea94938ce5/users/2da300340e074443a74e142d688b9ad5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:36.383122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.393843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72d8ce-e821-4d35-9dfc-ffc900805329 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.449874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.450543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.450830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.451108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.451856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.453498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b00c5-07f7-44a3-8be4-c55aa3349819 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.464002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1056/2115] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:36.468672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.479223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b95a6a-4f2c-41dd-ac90-6e2cce15f944 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.483806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.484715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.484994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.485267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] PATH_INFO: `/v3/projects/fee5ffe33cd94866a090a4ea94938ce5/users/2da300340e074443a74e142d688b9ad5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.487191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee5ffe33cd94866a090a4ea94938ce5, user_id=2da300340e074443a74e142d688b9ad5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.489884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.499887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1060/2116] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:36 2022] PUT /identity/v3/projects/fee5ffe33cd94866a090a4ea94938ce5/users/2da300340e074443a74e142d688b9ad5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:36.510852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.511316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.511405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.538375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] MFA Rules not processed for user `2da300340e074443a74e142d688b9ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:36.539712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:36.552625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:36.613584 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1057/2117] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:36.625761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:36.627702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:36.690068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7294974-8fdc-45c7-a19f-99c0bc7694bd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:36.694648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:36.694997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:36.695281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:36.695600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:36.717036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:36.756866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:36.758319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04e6e-7f3a-4fd7-a9b4-4a3b317a9843 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:36.767063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1061/2118] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:37.565947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:37.567373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:37.633015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c4b0d-89df-41ca-acc7-a559760cfb6d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:37.637570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:37.638435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:37.638721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:37.639001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] PATH_INFO: `/v3/users/e1b7e8915bb941b38398497f9a1d77ec` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:37.640182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b7e8915bb941b38398497f9a1d77ec)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:37.642155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:37.687388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b7e8915bb941b38398497f9a1d77ec'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:37.695027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b7e8915bb941b38398497f9a1d77ec'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:37.695699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b7e8915bb941b38398497f9a1d77ec'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:37.707819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b7e8915bb941b38398497f9a1d77ec'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:37.715601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b7e8915bb941b38398497f9a1d77ec'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:37.722186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1058/2119] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:37 2022] DELETE /identity/v3/users/e1b7e8915bb941b38398497f9a1d77ec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:37.728788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:37.729299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:37.729494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:37.747727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:37.749028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:37.826165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1062/2120] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:37.836730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:37.838236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:37.900282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:37.937158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e3df-16fc-4508-9408-e8278ca5d8cf admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:37.941130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:37.941827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:37.942107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:37.942519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:37.953556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:37.955153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:37.966004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1059/2121] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:38.235850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:38.237720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:38.257877 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190484a-f2c3-4f58-893a-d2bd4d2388bf None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:38.263216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:38.264094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:38.264383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:38.264742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] PATH_INFO: `/v3/projects/3d91fef56e7040d5a8c75b39fb8b1c34` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:38.266309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d91fef56e7040d5a8c75b39fb8b1c34)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:38.268519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:38.324269 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d91fef56e7040d5a8c75b39fb8b1c34', 'request_id': 'req-c81ff6e8-d009-441b-865d-b532f8b97908'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.331569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d91fef56e7040d5a8c75b39fb8b1c34', 'request_id': 'req-c81ff6e8-d009-441b-865d-b532f8b97908'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.339381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d91fef56e7040d5a8c75b39fb8b1c34', 'request_id': 'req-c81ff6e8-d009-441b-865d-b532f8b97908'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.350328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] The token cache is being invalidate because project 3d91fef56e7040d5a8c75b39fb8b1c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:38.350616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.351238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.352826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1063/2122] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:38 2022] DELETE /identity/v3/projects/3d91fef56e7040d5a8c75b39fb8b1c34 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:38.357248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:38.368010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b304f-f7fd-4251-acc0-e932701be061 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:38.424843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:38.425711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:38.425996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:38.426277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] PATH_INFO: `/v3/users/b56d43ede8da45c882d0e790b3e3eef6` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:38.428106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56d43ede8da45c882d0e790b3e3eef6)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:38.430085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:38.477724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56d43ede8da45c882d0e790b3e3eef6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.484588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56d43ede8da45c882d0e790b3e3eef6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.485272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56d43ede8da45c882d0e790b3e3eef6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.497681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56d43ede8da45c882d0e790b3e3eef6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.505082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6a203615-6009-4844-ad13-414a6110683b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56d43ede8da45c882d0e790b3e3eef6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.511138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1060/2123] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:38 2022] DELETE /identity/v3/users/b56d43ede8da45c882d0e790b3e3eef6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:38.784706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:38.786830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:38.801618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ff6e8-d009-441b-865d-b532f8b97908 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:38.851621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:38.869211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:38.870459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:38.870750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:38.871033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] PATH_INFO: `/v3/projects/dd0870af27dd4bdfb7348214ab636c18` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:38.872226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0870af27dd4bdfb7348214ab636c18)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:38.874199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:38.921754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0870af27dd4bdfb7348214ab636c18', 'request_id': 'req-7a9a68a6-e0cd-48d3-866a-823d228fc12d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.929026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0870af27dd4bdfb7348214ab636c18', 'request_id': 'req-7a9a68a6-e0cd-48d3-866a-823d228fc12d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.938326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0870af27dd4bdfb7348214ab636c18', 'request_id': 'req-7a9a68a6-e0cd-48d3-866a-823d228fc12d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.960851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] The token cache is being invalidate because project dd0870af27dd4bdfb7348214ab636c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:38.961229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.964838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7a9a68a6-e0cd-48d3-866a-823d228fc12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:38.971420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1064/2124] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:38 2022] DELETE /identity/v3/projects/dd0870af27dd4bdfb7348214ab636c18 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:38.973265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:38.973727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:38.973883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:38.994106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:38.995394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:39.045163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:39.060151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1061/2125] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:39.068768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.069174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.069520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.088132 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:39.089463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:39.097915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1065/2126] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:39.102585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.113503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4ca58-e403-41cf-99e2-b456ba6f36ca None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.119134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.119939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.120284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.120682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.121432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:39.121753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.123398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.136806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1062/2127] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:13:39.141210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.150484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef6920-031b-4f7f-a3e9-5f20f1c2ba64 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.155363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.156019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.156303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.156620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.157325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.159323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.186456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1066/2128] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:39.190754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.201900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed7483-ae0a-486f-98a0-bc4c60a53f9b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.258680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.259383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.259672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.259958 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.260736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.262670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.281686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1063/2129] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:39.285849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.296786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfcf1f-50c8-4904-a690-ba32b14377b2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.301690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.303213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.303566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.303846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.304642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.306256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.318268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1067/2130] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:39.322532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.333114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6802aa-0a50-4b57-8cf4-2b077fb20e55 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.337917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.338578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.338866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.339148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.339871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.341784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3774cb5-cebc-4e6a-b8e0-243291027bd0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.352309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1064/2131] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:39.356364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.367169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fda4b1-7811-4b55-8f1f-0f17be08da1c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.372270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.373218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.373489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.373760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] PATH_INFO: `/v3/projects/d371c1ac25364ca1979926841c0d2c7b/users/9bc1512ccbb74e04afc8cedcedc9a289/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.390637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d371c1ac25364ca1979926841c0d2c7b, user_id=9bc1512ccbb74e04afc8cedcedc9a289, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.393118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.406445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1068/2132] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:39 2022] PUT /identity/v3/projects/d371c1ac25364ca1979926841c0d2c7b/users/9bc1512ccbb74e04afc8cedcedc9a289/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:39.421826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.422120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.422413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.449273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] MFA Rules not processed for user `9bc1512ccbb74e04afc8cedcedc9a289`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:39.450476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:39.515567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1065/2133] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:39.520038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.581243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dea20f-03b1-42c9-a33a-096bc581aecb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.587069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.587725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.588386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.588698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.589381 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.591567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.607320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:39.620316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1069/2134] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:39.624534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.635263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0be76f-c3ec-4285-97af-86be41296aa0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.680527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:39.693044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.693792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.694081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.694360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.695056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.697017 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.717754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1066/2135] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:39.721841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.733392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b9642-f074-4c70-8a2a-0312a9d3c14a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.738103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.738763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.739052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.739464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.740330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.741983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.752416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1070/2136] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:39.756981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.766825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4974-308f-4f68-ad0a-9eb4c8e48396 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.772353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.773245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.773529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.773796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] PATH_INFO: `/v3/projects/34a42601b7364a3aa416763a9136975d/users/8093cfacf6274b6881872aedb3857f9b/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.789320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a42601b7364a3aa416763a9136975d, user_id=8093cfacf6274b6881872aedb3857f9b, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.791983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.805758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1067/2137] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:39 2022] PUT /identity/v3/projects/34a42601b7364a3aa416763a9136975d/users/8093cfacf6274b6881872aedb3857f9b/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:39.809785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.819942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e375-270f-4cdd-a9b8-83626fd9b7a6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.876167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.876782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.877066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.877339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.878051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.879640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48177027-620f-4f17-a262-c20b485df83c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.891517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1071/2138] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:39.895640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:39.906479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd0890-5604-42c5-90ee-cb45b8fa1d69 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:39.911152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:39.912095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.912380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.912703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] PATH_INFO: `/v3/projects/34a42601b7364a3aa416763a9136975d/users/8093cfacf6274b6881872aedb3857f9b/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.914846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a42601b7364a3aa416763a9136975d, user_id=8093cfacf6274b6881872aedb3857f9b, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:39.917567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:39.928420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1068/2139] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:39 2022] PUT /identity/v3/projects/34a42601b7364a3aa416763a9136975d/users/8093cfacf6274b6881872aedb3857f9b/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:39.939335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:39.939633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:39.939922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:39.967169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] MFA Rules not processed for user `8093cfacf6274b6881872aedb3857f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:39.968470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:40.042157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1072/2140] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:40.053055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:40.054808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:40.111204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2155c6b-be70-4bcb-ae18-f724dae88fd1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:40.115627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:40.116277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:40.116589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:40.116870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:40.125916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:40.127394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002817dc-19df-4357-821c-78039d490f5e service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:40.137496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1069/2141] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:40.267084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:40.335778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fabd89-f666-42a1-9ee9-350823379f5c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:40.340516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:40.341629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:40.341904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:40.342174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] PATH_INFO: `/v3/users/9bc1512ccbb74e04afc8cedcedc9a289` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:40.343357 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc1512ccbb74e04afc8cedcedc9a289)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:40.345342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:40.351985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:40.399142 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc1512ccbb74e04afc8cedcedc9a289'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.405924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc1512ccbb74e04afc8cedcedc9a289'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.406717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc1512ccbb74e04afc8cedcedc9a289'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.419354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc1512ccbb74e04afc8cedcedc9a289'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.419986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc1512ccbb74e04afc8cedcedc9a289'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.432092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1073/2142] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:40 2022] DELETE /identity/v3/users/9bc1512ccbb74e04afc8cedcedc9a289 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:40.436762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:40.437063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:40.437351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:40.455267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:40.456627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:40.516866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1070/2143] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:40.527445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:40.528886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:40.585914 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ec40b-d002-44e8-83cc-04dfdc56ff62 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:40.589629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:40.590298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:40.590609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:40.591166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:40.601006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:40.602449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:40.611782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1074/2144] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:40.879159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:40.881354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:40.894657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9dd73-7318-4e34-906a-50c4f4203999 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:40.900276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:40.901182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:40.901468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:40.901746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] PATH_INFO: `/v3/projects/d371c1ac25364ca1979926841c0d2c7b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:40.902940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d371c1ac25364ca1979926841c0d2c7b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:40.904994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:40.947174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:40.956407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd371c1ac25364ca1979926841c0d2c7b', 'request_id': 'req-61c3302c-c1c3-4402-915c-1ddad852016c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.963371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd371c1ac25364ca1979926841c0d2c7b', 'request_id': 'req-61c3302c-c1c3-4402-915c-1ddad852016c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.969851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd371c1ac25364ca1979926841c0d2c7b', 'request_id': 'req-61c3302c-c1c3-4402-915c-1ddad852016c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.982560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] The token cache is being invalidate because project d371c1ac25364ca1979926841c0d2c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:40.982850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.983470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:40.985050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1071/2145] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:40 2022] DELETE /identity/v3/projects/d371c1ac25364ca1979926841c0d2c7b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:40.989057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:40.998812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc87533-3ec4-456b-9458-2267599bb8e8 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.058794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.059670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.059959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.060240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] PATH_INFO: `/v3/users/8093cfacf6274b6881872aedb3857f9b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.061527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8093cfacf6274b6881872aedb3857f9b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.063461 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.097782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:41.112791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8093cfacf6274b6881872aedb3857f9b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.121068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8093cfacf6274b6881872aedb3857f9b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.121670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8093cfacf6274b6881872aedb3857f9b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.133892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8093cfacf6274b6881872aedb3857f9b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.134511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f0d5e1ce-e838-4395-a609-46c07dadcd0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8093cfacf6274b6881872aedb3857f9b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.146491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1075/2146] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:40 2022] DELETE /identity/v3/users/8093cfacf6274b6881872aedb3857f9b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:41.432373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.434728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:41.449583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3302c-c1c3-4402-915c-1ddad852016c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.513893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.514793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.515073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.515346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] PATH_INFO: `/v3/projects/34a42601b7364a3aa416763a9136975d` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.516580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a42601b7364a3aa416763a9136975d)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.518514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.573825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a42601b7364a3aa416763a9136975d', 'request_id': 'req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.580657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a42601b7364a3aa416763a9136975d', 'request_id': 'req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.587803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a42601b7364a3aa416763a9136975d', 'request_id': 'req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.603737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] The token cache is being invalidate because project 34a42601b7364a3aa416763a9136975d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:41.604035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.605287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-1fc1a2bb-d7d6-4bd0-a8ea-d96f3c309819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:41.606773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1072/2147] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:41 2022] DELETE /identity/v3/projects/34a42601b7364a3aa416763a9136975d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:41.611675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.612047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.612412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.634210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:41.635431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:41.696792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1076/2148] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:41.704421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.704747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.705029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.725078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:41.726321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:41.734618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1073/2149] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:41.738730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.748466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc6245-e373-481e-ae95-ac7414fa8e48 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.752939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.753583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.753864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.754139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.754856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:41.755751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.757562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.768955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1077/2150] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:41.773708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.782964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567742-6888-415c-86b0-4d1a41aca25d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.787857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.788626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.789008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.789294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.789993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.792137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.824297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1074/2151] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:41 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:41.828917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.839961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce437b-0c2b-431a-ab52-822211179722 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.897208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.897872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.898161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.898442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.899143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.901127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.921610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1078/2152] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:41 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:41.925591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.936254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc55e78-1a2c-466f-99b7-b5b201b2685e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.941012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.941670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.941949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.942224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.943899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.945549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.955911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1075/2153] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:41.959904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:41.970105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740fe259-45de-4b7e-a706-c727ad1daf2e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:41.975587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:41.976237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:41.976548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:41.976826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:41.977543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:41.979150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7dbefa-6ced-4748-99bd-48688dff35c0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:41.988917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1079/2154] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:41.993703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.003694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daabf25-c6a9-4adf-a9d1-c1ba9fb45d67 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.009012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.009886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.010174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.010455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] PATH_INFO: `/v3/projects/91bc5fc3b229407c9537490e6361119b/users/6c0f7915d7514d289ec75e91e141c75b/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.025055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91bc5fc3b229407c9537490e6361119b, user_id=6c0f7915d7514d289ec75e91e141c75b, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.027845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.030071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:42.042979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1076/2155] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:41 2022] PUT /identity/v3/projects/91bc5fc3b229407c9537490e6361119b/users/6c0f7915d7514d289ec75e91e141c75b/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:42.053906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.054350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.054544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.079965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:42.083803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] MFA Rules not processed for user `6c0f7915d7514d289ec75e91e141c75b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:42.085231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:42.169553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1080/2156] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:42.174593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.240852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821d36-9d05-49cb-87d4-6a4351a9f1a2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.245552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.246218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.246505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.246786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.247481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.249444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.282210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1077/2157] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:42.285332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.297015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71acf54-beea-4640-912d-22d5663638a5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.356618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.357299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.357582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.357858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.358582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.360495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.388689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1081/2158] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:13:42.392346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.403106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4670a-507f-4a3f-80a5-0fd850b82cbd admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.407763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.408424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.408763 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.409039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.409761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.411974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.432976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1078/2159] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:42.433520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.440479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b385-719c-4050-9edd-db553527dc68 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.446369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.447034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.447320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.447599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.448325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.449974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae442c-dab4-4877-8785-cd7fe425b1ec admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.460945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1082/2160] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:42.465251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.476496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcdea3-643b-4f27-9d05-92cb2598a5e9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.481599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.482638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.482981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.483331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] PATH_INFO: `/v3/projects/d49131cdfe68437ba5127e69876bce1b/users/d819742e96e04effbe2a18d6f3acf436/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.499441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49131cdfe68437ba5127e69876bce1b, user_id=d819742e96e04effbe2a18d6f3acf436, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.502270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.517253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1079/2161] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:42 2022] PUT /identity/v3/projects/d49131cdfe68437ba5127e69876bce1b/users/d819742e96e04effbe2a18d6f3acf436/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:42.528804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.529103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.529388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.560266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] MFA Rules not processed for user `d819742e96e04effbe2a18d6f3acf436`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:42.560266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:42.625082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1083/2162] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:42.629636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.695483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215672-300b-4c14-a5f9-7715a78c66a4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.700405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.701103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.701406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.701833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.702640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.704566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.737295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1080/2163] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:42.742230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.753848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230f46d-c9fc-4ca4-a808-df2cd2fbc31c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.768401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:42.829406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.830620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.830980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.831336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.832211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.835155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.865783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1084/2164] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:42 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:42.869949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.880879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43f8d9-02c1-47e1-acec-2c95f108ac25 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.885635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.886276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.886562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.886951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.887685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.889588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.900248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1081/2165] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:42.904312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.914834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11f048-c618-4f39-abb3-5238d04cc812 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.920393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:42.921289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:42.921571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:42.921844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] PATH_INFO: `/v3/projects/c352678f42714919b56abb5d4d066adc/users/08c30e0440b142a5a9d5d38df1296525/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:42.937321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c352678f42714919b56abb5d4d066adc, user_id=08c30e0440b142a5a9d5d38df1296525, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:42.939965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:42.953541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1085/2166] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:42 2022] PUT /identity/v3/projects/c352678f42714919b56abb5d4d066adc/users/08c30e0440b142a5a9d5d38df1296525/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:42.957351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:42.968057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8c62-56c8-4290-88bc-0c9e4f71df47 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:42.981075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:43.026438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:43.027106 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:43.027394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:43.027676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:43.028398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:43.030035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b92dad-93e7-4ba3-a6f3-83efac588c38 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:43.040252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1082/2167] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:43.044409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:43.055405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4deba8-5849-441b-9b86-c9bffb314007 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:43.059970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:43.060878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:43.061162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:43.061436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] PATH_INFO: `/v3/projects/c352678f42714919b56abb5d4d066adc/users/08c30e0440b142a5a9d5d38df1296525/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:43.063379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c352678f42714919b56abb5d4d066adc, user_id=08c30e0440b142a5a9d5d38df1296525, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:43.066066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:43.076399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1086/2168] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:43 2022] PUT /identity/v3/projects/c352678f42714919b56abb5d4d066adc/users/08c30e0440b142a5a9d5d38df1296525/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:43.086534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:43.086824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:43.087104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:43.115623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] MFA Rules not processed for user `08c30e0440b142a5a9d5d38df1296525`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:43.116888 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:43.189434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1083/2169] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:43.196577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:43.198515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:43.257519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c39bac-9b58-4b05-a567-39bbfc8ca28b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:43.261097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:43.261740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:43.262028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:43.262308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:43.271386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:43.272871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:43.282385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1087/2170] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:43.323649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:43.325063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:43.337843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e777d-413d-4c91-ba3f-f30fda56bcf4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:43.341734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:43.342375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:43.342661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:43.342941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:43.400831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:43.402282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:43.411925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1084/2171] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:45.534053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:45.543191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2732c-5d18-463e-9c02-4947c98acc8c service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:45.546803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:45.547434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:45.547721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:45.548001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:45.549784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:45.617189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:45.618656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538bb37-a00a-4ba9-a12e-15d44a6f5e3f service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:45.624439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:45.629976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1088/2172] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:53.407418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=83 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:53.695020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:53.815582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f3a0-2f9c-4f49-84c8-60b28973eaa6 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:53.821607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:53.822581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:53.822755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:53.823036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] PATH_INFO: `/v3/users/6c0f7915d7514d289ec75e91e141c75b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:53.824261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c0f7915d7514d289ec75e91e141c75b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:53.826400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:53.883114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c0f7915d7514d289ec75e91e141c75b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:53.892211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c0f7915d7514d289ec75e91e141c75b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:53.892959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c0f7915d7514d289ec75e91e141c75b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:53.905792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c0f7915d7514d289ec75e91e141c75b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:53.913678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c0f7915d7514d289ec75e91e141c75b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:53.920185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1085/2173] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:53 2022] DELETE /identity/v3/users/6c0f7915d7514d289ec75e91e141c75b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:53.924928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:53.925416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:53.925614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:53.983022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:53.984309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.044155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1089/2174] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:54.054627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:54.056163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.098866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:54.152952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fcc85-2ab9-4531-a87d-e721317a4c38 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:54.156586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:54.157233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.157537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.157799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.166876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:54.168299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:54.178195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1086/2175] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:54.440971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:54.443124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.455169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cff6ba-1bbc-4f9d-bccf-e394ed60280e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:54.460927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:54.461864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.462154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.462433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] PATH_INFO: `/v3/projects/91bc5fc3b229407c9537490e6361119b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.463640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=91bc5fc3b229407c9537490e6361119b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:54.465690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:54.495732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:54.497299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.510899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16ae9f-0c04-4e49-aa3a-31b138a139ef service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:54.516732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91bc5fc3b229407c9537490e6361119b', 'request_id': 'req-2803e908-a810-47b1-8bab-c7b14fac2bfb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.523837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91bc5fc3b229407c9537490e6361119b', 'request_id': 'req-2803e908-a810-47b1-8bab-c7b14fac2bfb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.531673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91bc5fc3b229407c9537490e6361119b', 'request_id': 'req-2803e908-a810-47b1-8bab-c7b14fac2bfb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.543337 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] The token cache is being invalidate because project 91bc5fc3b229407c9537490e6361119b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:54.543738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.544460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.546095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1090/2176] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:54 2022] DELETE /identity/v3/projects/91bc5fc3b229407c9537490e6361119b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:54.549856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:54.551249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.571010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:54.571967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.572259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.572577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] PATH_INFO: `/v3/users/d3472b6d287a493b9239175512b6d9cc` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.573773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3472b6d287a493b9239175512b6d9cc)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:54.576201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:54.617711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:54.621976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803e908-a810-47b1-8bab-c7b14fac2bfb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:54.634250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:54.634250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.634250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.634250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] PATH_INFO: `/v3/users/d819742e96e04effbe2a18d6f3acf436` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.634600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d819742e96e04effbe2a18d6f3acf436)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:54.635197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:54.636472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3472b6d287a493b9239175512b6d9cc'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.644001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3472b6d287a493b9239175512b6d9cc'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.644713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3472b6d287a493b9239175512b6d9cc'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.657497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3472b6d287a493b9239175512b6d9cc'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.665910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-485d08f6-5e0a-4e5d-8502-c2d0a9577034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3472b6d287a493b9239175512b6d9cc'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.672637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1087/2177] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:54 2022] DELETE /identity/v3/users/d3472b6d287a493b9239175512b6d9cc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:54.677178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.677540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.677905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.684184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd819742e96e04effbe2a18d6f3acf436'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.691733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd819742e96e04effbe2a18d6f3acf436'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.692888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd819742e96e04effbe2a18d6f3acf436'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.696198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:54.697457 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.705759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd819742e96e04effbe2a18d6f3acf436'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.706503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd819742e96e04effbe2a18d6f3acf436'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:54.717142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:54.720955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1091/2178] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:54 2022] DELETE /identity/v3/users/d819742e96e04effbe2a18d6f3acf436 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:54.763418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1088/2179] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:13:54.773660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:54.775124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:54.835978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334bb9a-af30-4996-9b74-5316aa5e89eb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:54.839654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:54.840300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:54.840626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:54.841160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:54.909289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:54.910996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:54.923124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1092/2180] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:13:55.267079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:55.268523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:55.282514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d278d-9c3f-43fd-ac4e-6501fae90fc1 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:55.291973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:55.292915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:55.293196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:55.293522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] PATH_INFO: `/v3/projects/d49131cdfe68437ba5127e69876bce1b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:55.294924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49131cdfe68437ba5127e69876bce1b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:55.297623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:55.353754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49131cdfe68437ba5127e69876bce1b', 'request_id': 'req-4c1ef543-e6c7-448b-923c-31cb157346c6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.360364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49131cdfe68437ba5127e69876bce1b', 'request_id': 'req-4c1ef543-e6c7-448b-923c-31cb157346c6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.366146 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49131cdfe68437ba5127e69876bce1b', 'request_id': 'req-4c1ef543-e6c7-448b-923c-31cb157346c6'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.384010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] The token cache is being invalidate because project d49131cdfe68437ba5127e69876bce1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:55.384308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.384997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.386837 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1089/2181] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:55 2022] DELETE /identity/v3/projects/d49131cdfe68437ba5127e69876bce1b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:55.390607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:55.400922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740322d-963d-4b4a-90b1-bea000467119 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:55.425602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:55.462976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:55.464693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:55.468630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:55.469588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:55.469866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:55.470226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] PATH_INFO: `/v3/users/08c30e0440b142a5a9d5d38df1296525` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:55.471488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c30e0440b142a5a9d5d38df1296525)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:55.473680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:55.482167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ef543-e6c7-448b-923c-31cb157346c6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:55.487553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:55.488473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:55.488732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:55.490000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] PATH_INFO: `/v3/projects/9b5a8f5cf5694164bd6bfa640178de4b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:55.491514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5a8f5cf5694164bd6bfa640178de4b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:55.493547 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:55.527509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c30e0440b142a5a9d5d38df1296525'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.534484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c30e0440b142a5a9d5d38df1296525'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.535172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c30e0440b142a5a9d5d38df1296525'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.544809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5a8f5cf5694164bd6bfa640178de4b', 'request_id': 'req-f3034adf-b5aa-4944-b461-f021e52f1999'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.547495 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c30e0440b142a5a9d5d38df1296525'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.548615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c30e0440b142a5a9d5d38df1296525'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.551421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5a8f5cf5694164bd6bfa640178de4b', 'request_id': 'req-f3034adf-b5aa-4944-b461-f021e52f1999'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.558861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5a8f5cf5694164bd6bfa640178de4b', 'request_id': 'req-f3034adf-b5aa-4944-b461-f021e52f1999'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.562290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1093/2182] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:13:55 2022] DELETE /identity/v3/users/08c30e0440b142a5a9d5d38df1296525 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:55.578424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] The token cache is being invalidate because project 9b5a8f5cf5694164bd6bfa640178de4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:55.578716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.579351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f3034adf-b5aa-4944-b461-f021e52f1999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:55.580957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1090/2183] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:55 2022] DELETE /identity/v3/projects/9b5a8f5cf5694164bd6bfa640178de4b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:55.867642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:55.869123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:55.937444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff2307-ef6f-4505-9df5-9594b2200b61 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:55.942034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:55.943254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:55.943542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:55.943818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] PATH_INFO: `/v3/projects/c352678f42714919b56abb5d4d066adc` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:55.945057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c352678f42714919b56abb5d4d066adc)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:55.946995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:55.996082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:55.997321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c352678f42714919b56abb5d4d066adc', 'request_id': 'req-df956ae4-c708-4f8e-951e-4ea0bf8088f4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:56.005670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c352678f42714919b56abb5d4d066adc', 'request_id': 'req-df956ae4-c708-4f8e-951e-4ea0bf8088f4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:56.012856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c352678f42714919b56abb5d4d066adc', 'request_id': 'req-df956ae4-c708-4f8e-951e-4ea0bf8088f4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:56.023136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] The token cache is being invalidate because project c352678f42714919b56abb5d4d066adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:13:56.023428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:56.024057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-df956ae4-c708-4f8e-951e-4ea0bf8088f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:13:56.025625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1094/2184] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:13:55 2022] DELETE /identity/v3/projects/c352678f42714919b56abb5d4d066adc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:56.031466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.031758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.032068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.071565 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:56.072912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:56.083540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:56.142062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1091/2185] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:56.152004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.152302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.152636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.171246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:56.172510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:56.180916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1095/2186] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:56.185053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.194903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e341a1-b23f-48aa-83d4-8a225445fd00 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.199560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.200212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.200530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.200812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.201546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:13:56.201857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.204204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.216991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1092/2187] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:13:56.221351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.231283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17a062-1748-4257-a38e-b3d8e4ba00ce None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.235969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.236644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.236938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.237218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.237916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.239884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.264427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1096/2188] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:56 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:56.268304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.279137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4388d-f100-4510-ba3c-5424df9ed92a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.335608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.336476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.336592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.336841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.337554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.339515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.359820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1093/2189] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:56 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:56.363874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.374416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b980b52-20e2-46d8-b66f-0732ebc158aa admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.379266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.380067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.380419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.380729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.381459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.383094 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.396756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1097/2190] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:56.401398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.412138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51fab-668f-4f38-9d54-bf78fd64494d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.416898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.417549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.417831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.418109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.418829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.420417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5497ea-0eb8-403c-934d-01c0c0252bd6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.430756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1094/2191] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:56.434757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.445600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe196cf-d5ec-4b5b-a43b-6e38b9df5c37 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.450241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.451110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.451417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.451776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] PATH_INFO: `/v3/projects/17c8de06d59a4968a103f8f72d6a501f/users/fce3221c478e4addb801a8581928bbc2/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.467401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c8de06d59a4968a103f8f72d6a501f, user_id=fce3221c478e4addb801a8581928bbc2, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.470128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.482521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1098/2192] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:56 2022] PUT /identity/v3/projects/17c8de06d59a4968a103f8f72d6a501f/users/fce3221c478e4addb801a8581928bbc2/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:56.487559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25792c85-d6b3-44ec-b344-fc4f95b2bfbd None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.487852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25792c85-d6b3-44ec-b344-fc4f95b2bfbd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.488135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25792c85-d6b3-44ec-b344-fc4f95b2bfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.507116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-25792c85-d6b3-44ec-b344-fc4f95b2bfbd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:56.508325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-25792c85-d6b3-44ec-b344-fc4f95b2bfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:56.566907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1095/2193] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:56.577048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:56.578702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:56.636032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b20c-5c99-40cb-909a-6325fc7d6d11 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:56.639599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:56.640238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:56.640576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:56.640854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:56.650527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:56.651964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:56.661396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1099/2194] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:57.285404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.285703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.286508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.313753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] MFA Rules not processed for user `fce3221c478e4addb801a8581928bbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:57.314977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:57.324786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:57.386478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1096/2195] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:57.391306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.401959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70475f58-fb21-48bc-b3d7-a09b0956dd18 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.406723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.407529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.407880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.408222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.409028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.411952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.440212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1100/2196] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:13:57 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:57.445005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.456162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b8985-9e0e-4114-a475-94db89b75a3c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.520941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.521626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.521908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.522206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.522920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.524987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.545396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1097/2197] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:13:57 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:13:57.549521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.560585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed726c69-4452-410d-a4fc-411088cbcb77 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.566133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.566931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.567623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.568121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.569130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.570972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.589818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1101/2198] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:57.595154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.606501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4abf0e-372b-466e-a9cb-c79c0d912f4c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.611211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.612092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.612381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.612701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] PATH_INFO: `/v3/projects/aa4a2a9d25a34e73b8202cece862c68b/users/d5612c703f9e4bac9cf71d46adc1500a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.628834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4a2a9d25a34e73b8202cece862c68b, user_id=d5612c703f9e4bac9cf71d46adc1500a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.632201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.644662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1098/2199] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:57 2022] PUT /identity/v3/projects/aa4a2a9d25a34e73b8202cece862c68b/users/d5612c703f9e4bac9cf71d46adc1500a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:13:57.649450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.659226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a8d3b-4f95-41fb-a7b7-5eda1ab979a8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.702587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:57.722775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.724266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.724664 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.725011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.725815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.727468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3dcf0-e038-4e3d-9e61-cf75ca7c100b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.739671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1102/2200] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:13:57.742964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:57.754102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517698c4-fee9-4957-b25b-5b001daa0606 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:57.759476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:57.760666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:57.761033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:57.761389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] PATH_INFO: `/v3/projects/aa4a2a9d25a34e73b8202cece862c68b/users/d5612c703f9e4bac9cf71d46adc1500a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:57.763859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4a2a9d25a34e73b8202cece862c68b, user_id=d5612c703f9e4bac9cf71d46adc1500a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:57.766432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:57.780284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1099/2201] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:13:57 2022] PUT /identity/v3/projects/aa4a2a9d25a34e73b8202cece862c68b/users/d5612c703f9e4bac9cf71d46adc1500a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:13:58.617401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:58.617952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:58.617952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:58.679212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] MFA Rules not processed for user `d5612c703f9e4bac9cf71d46adc1500a`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:58.680501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:58.776477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1103/2202] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:58.789533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:58.791643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:58.852334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b068-6f98-4d59-9e48-142afce5f237 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:58.856044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:58.856720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:58.857002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:58.857328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:58.875699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:58.919475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:58.921087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:58.932049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1100/2203] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:13:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:03.506904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:03.508290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:03.520786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757dc3-8766-40c0-aec9-aab76cb07d46 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:03.524362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:03.525140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:03.525426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:03.525703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:03.534653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:03.536642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f7dbf-13c8-4dd8-878c-d71680660459 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:03.546583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1104/2204] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:08.927669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=19 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:13.024303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:13.025781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.092274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788269ac-367f-4713-8c1f-2b8944cc9b96 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:13.097717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:13.098614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.098922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.099204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] PATH_INFO: `/v3/users/250b237759c44a688dfd4aa685ec4d8b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.100965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=250b237759c44a688dfd4aa685ec4d8b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:13.102897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:13.149556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250b237759c44a688dfd4aa685ec4d8b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.156372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250b237759c44a688dfd4aa685ec4d8b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.157141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250b237759c44a688dfd4aa685ec4d8b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.169588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250b237759c44a688dfd4aa685ec4d8b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.176778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250b237759c44a688dfd4aa685ec4d8b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.183444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1101/2205] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:13 2022] DELETE /identity/v3/users/250b237759c44a688dfd4aa685ec4d8b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:13.187981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.188531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.188627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.206360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:13.207593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.266518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1105/2206] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:13.276908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:13.278334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.335762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65dd7a-4f6f-4ade-bc83-eb70a46e11ce admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:13.339426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:13.340071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.340353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.340655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.350511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:13.352078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b3eba-0a1d-4cd4-b58b-74a0920ea5eb service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:13.361596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1102/2207] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:13.653284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:13.655103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.672274 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2cd14-00e4-4e24-9d83-2ef320d1c520 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:13.677515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:13.679140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.679783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.680259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] PATH_INFO: `/v3/projects/09b8cd4a676c4ebab41612617d76321b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.681817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b8cd4a676c4ebab41612617d76321b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:13.683940 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:13.760935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:13.766294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b8cd4a676c4ebab41612617d76321b', 'request_id': 'req-49a2680e-c6a5-459f-95a1-748318c5acce'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.780032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b8cd4a676c4ebab41612617d76321b', 'request_id': 'req-49a2680e-c6a5-459f-95a1-748318c5acce'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.791294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b8cd4a676c4ebab41612617d76321b', 'request_id': 'req-49a2680e-c6a5-459f-95a1-748318c5acce'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.807432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] The token cache is being invalidate because project 09b8cd4a676c4ebab41612617d76321b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:13.807728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.808541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-49a2680e-c6a5-459f-95a1-748318c5acce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:13.810149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1106/2208] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:13 2022] DELETE /identity/v3/projects/09b8cd4a676c4ebab41612617d76321b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:13.815115 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.815411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.815722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.851611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:13.852850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.862117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:13.922511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1103/2209] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:13.930659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.930952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.931774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.951104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:13.952349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:13.961989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1107/2210] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:13.966339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:13.985206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215af78-956f-46e9-b6e0-efe790e55b3a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:13.990096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:13.992323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:13.993142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:13.993425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:13.994162 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:13.994472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:13.996064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.011836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1104/2211] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:14:14.013309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.023758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88749-a7ef-42df-b984-ac7929514c5d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.028468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.029354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.029626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.029992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.030803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.032792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.061573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1108/2212] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:14 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:14.065419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.077109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934eca3-f78a-4cfe-8abf-02005ba03b34 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.137227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.137895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.138181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.138460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.139154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.141117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.160132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1105/2213] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:14 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:14.164437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.174937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8774-193b-488b-9348-030dbbc31795 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.180276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.180952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.181239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.181517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.182247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.183856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.195058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1109/2214] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:14.199200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.210614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bcd85-5cb1-4234-9a2f-8954725ea656 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.215445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.216369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.216700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.216973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.217695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.219308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2059b9c-f50c-4033-8582-1edef2fa966b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.229905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1106/2215] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:14.234368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.245086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11067d-1d0b-4f1d-abe6-0dce8a0d8e2c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.249788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.250682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.250970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.251250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] PATH_INFO: `/v3/projects/ceb79f09ffc544199432d8627b2d1bf4/users/59de120fab9f477f9a48a0b6f2af927a/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.268735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb79f09ffc544199432d8627b2d1bf4, user_id=59de120fab9f477f9a48a0b6f2af927a, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.271486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.285259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1110/2216] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:14 2022] PUT /identity/v3/projects/ceb79f09ffc544199432d8627b2d1bf4/users/59de120fab9f477f9a48a0b6f2af927a/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:14.296242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.296562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.296846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.325097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] MFA Rules not processed for user `59de120fab9f477f9a48a0b6f2af927a`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:14.326313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:14.391095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1107/2217] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:14.402537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.404285 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:14.461055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba21633-5d30-4982-822d-238cb4a8cf40 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.464655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.465290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.465568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.465841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.474445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.476437 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245894dc-b0eb-46f3-96e9-dce89b73670b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.485092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1111/2218] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:14.729904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.731593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:14.744159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:14.806887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc2da3-36c8-4f65-bd5a-83cf9a563bd9 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:14.811468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:14.812337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.812765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.812919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] PATH_INFO: `/v3/users/1a8d7ced0251440699c0d9ffe413459a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.814120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a8d7ced0251440699c0d9ffe413459a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:14.816035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:14.864092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a8d7ced0251440699c0d9ffe413459a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:14.871392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a8d7ced0251440699c0d9ffe413459a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:14.872064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a8d7ced0251440699c0d9ffe413459a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:14.884256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a8d7ced0251440699c0d9ffe413459a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:14.891781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a8d7ced0251440699c0d9ffe413459a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:14.899527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1108/2219] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:14 2022] DELETE /identity/v3/users/1a8d7ced0251440699c0d9ffe413459a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:14.903857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:14.904220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:14.904653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:14.922314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:14.923600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:14.987510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1112/2220] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:14.997400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:14.998807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:15.055638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ce82-d423-4bd8-b1a5-7939c4a69a54 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:15.059368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:15.059949 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:15.060232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:15.060538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:15.069553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:15.070985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:15.081668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1109/2221] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:15.363183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:15.364702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:15.384800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed411c-d6eb-42f5-9823-f0518f15e784 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:15.390993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:15.391875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:15.392164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:15.392494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] PATH_INFO: `/v3/projects/c88d1ec683574cadabfa671988b58f90` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:15.393700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88d1ec683574cadabfa671988b58f90)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:15.395647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:15.405245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:15.465214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88d1ec683574cadabfa671988b58f90', 'request_id': 'req-268342ca-95ed-4191-beeb-1b0a22c4c3d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.472088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88d1ec683574cadabfa671988b58f90', 'request_id': 'req-268342ca-95ed-4191-beeb-1b0a22c4c3d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.480778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88d1ec683574cadabfa671988b58f90', 'request_id': 'req-268342ca-95ed-4191-beeb-1b0a22c4c3d9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.491097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] The token cache is being invalidate because project c88d1ec683574cadabfa671988b58f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:15.491385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.492136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.493836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1113/2222] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:15 2022] DELETE /identity/v3/projects/c88d1ec683574cadabfa671988b58f90 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:15.498303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:15.509218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219618d9-3ac5-4096-99f4-25ae76938bf3 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:15.539524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:15.569554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:15.570417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:15.570695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:15.570971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] PATH_INFO: `/v3/users/2da300340e074443a74e142d688b9ad5` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:15.572147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da300340e074443a74e142d688b9ad5)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:15.574102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:15.621070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da300340e074443a74e142d688b9ad5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.628355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da300340e074443a74e142d688b9ad5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.629040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da300340e074443a74e142d688b9ad5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.641539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da300340e074443a74e142d688b9ad5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.648805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5ec1c804-4cc7-45b7-acde-7717a344ce87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da300340e074443a74e142d688b9ad5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:15.655015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1110/2223] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:15 2022] DELETE /identity/v3/users/2da300340e074443a74e142d688b9ad5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:15.945651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:15.949316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:15.963551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268342ca-95ed-4191-beeb-1b0a22c4c3d9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.035490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.036272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.036471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.036756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] PATH_INFO: `/v3/projects/fee5ffe33cd94866a090a4ea94938ce5` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.038720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee5ffe33cd94866a090a4ea94938ce5)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.041138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.107239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee5ffe33cd94866a090a4ea94938ce5', 'request_id': 'req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:16.114532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee5ffe33cd94866a090a4ea94938ce5', 'request_id': 'req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:16.126708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee5ffe33cd94866a090a4ea94938ce5', 'request_id': 'req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:16.146875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] The token cache is being invalidate because project fee5ffe33cd94866a090a4ea94938ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:16.147698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:16.148948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60a7eaf-f00a-47a1-a964-8f0a11b736b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:16.152161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1114/2224] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:15 2022] DELETE /identity/v3/projects/fee5ffe33cd94866a090a4ea94938ce5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:16.158315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.159058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.159058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.179753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:16.181019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:16.266873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1111/2225] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:16.276595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.276893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.277177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.312353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:16.313811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:16.322490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1115/2226] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:16.327931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.338592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f33c3-0e25-46ff-a1b5-f337da34336d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.346356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.346356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.346356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.346356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.346786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:16.346972 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.348563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.372484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1112/2227] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:14:16.374277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.388621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143268b-e72a-4837-a48f-7a0bdcfd0f99 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.393552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.394321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.394612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.394904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.395796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.398666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.419054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:16.429154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1116/2228] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:16.433349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.443899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc056-2f1f-47da-94f6-40a4a0a4f150 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.479817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:16.508403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.509109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.509395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.509674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.510369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.512295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.533184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1113/2229] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:14:16.537364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.547678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32a603-c5d4-4f21-b7ae-cff79b6cb37e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.553187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.554066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.554356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.554640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.555367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.557133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.567834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1117/2230] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:16.572197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.582532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa5319-6c18-4c66-a713-4401fd3d062f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.589026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.589702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.589987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.590288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.591251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.593249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be905959-f67d-4205-908d-cd80ab020382 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.605185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1114/2231] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:16.609374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.620552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25ed5a-6f84-454f-aa31-96d82c7e5aff admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.625293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.626265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.626462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.626738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] PATH_INFO: `/v3/projects/b08b129b334b4f77b9d0f31e9c970797/users/9209496c805a46d5b28427ef4b9d5f7f/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.642785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08b129b334b4f77b9d0f31e9c970797, user_id=9209496c805a46d5b28427ef4b9d5f7f, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.645592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.659962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1118/2232] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:16 2022] PUT /identity/v3/projects/b08b129b334b4f77b9d0f31e9c970797/users/9209496c805a46d5b28427ef4b9d5f7f/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:16.670866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.671212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.671496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.702733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] MFA Rules not processed for user `9209496c805a46d5b28427ef4b9d5f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:16.704068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:16.773923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1115/2233] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:16.778433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.840413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15533f3-cf1d-4043-98e1-9f68595824a3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.845489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.846137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.846424 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.846784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.847453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.850041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.877539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1119/2234] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:16.881933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.893153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4458eb1-1adc-44b8-b2b6-ed5d51c41d8c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.953830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.954495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.954781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.955058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:16.955740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:16.957706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:16.978169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1116/2235] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:16 2022] POST /identity/v3/users => generated 306 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:16.982190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:16.993470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf55b6-460f-46e7-9944-06bf2dfe32c5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:16.998429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:16.999069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:16.999355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:16.999655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.000386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.002036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.013710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1120/2236] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:17.017832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:17.028306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23358b25-5605-4e92-864e-2434bdc0c8b1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:17.032910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:17.033778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.034063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.034342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] PATH_INFO: `/v3/projects/c984683e6cc4404fa79fc5101fa5aab1/users/683593da3655461aad4e789e09324fd3/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.048766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c984683e6cc4404fa79fc5101fa5aab1, user_id=683593da3655461aad4e789e09324fd3, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.051435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.064801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1117/2237] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:17 2022] PUT /identity/v3/projects/c984683e6cc4404fa79fc5101fa5aab1/users/683593da3655461aad4e789e09324fd3/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:17.069153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:17.079510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e16f6-4924-4dd1-9ee5-6860feb69150 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:17.137459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:17.138155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.138441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.138722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.139468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.141127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d9470-3dbe-4c81-90d6-c40185c53208 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.151308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1121/2238] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:17.155434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:17.166170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035783-8dc2-4352-a464-e07875fe4a5a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:17.170796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:17.171599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.171879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.172155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] PATH_INFO: `/v3/projects/c984683e6cc4404fa79fc5101fa5aab1/users/683593da3655461aad4e789e09324fd3/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.174102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c984683e6cc4404fa79fc5101fa5aab1, user_id=683593da3655461aad4e789e09324fd3, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.176768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.182512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:17.189117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1118/2239] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:17 2022] PUT /identity/v3/projects/c984683e6cc4404fa79fc5101fa5aab1/users/683593da3655461aad4e789e09324fd3/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:17.199431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.199725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.200010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.230517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] MFA Rules not processed for user `683593da3655461aad4e789e09324fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:17.231709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:17.252303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:17.307500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1122/2240] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:17.318562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:17.320950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:17.391149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158475cb-92b7-4e56-aec7-e0ff776fcbc5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:17.395370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:17.396025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.396406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.397372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.408271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.410071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.420907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1119/2241] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:17.505487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:17.516471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705eced-6159-4516-a8d2-afac1f29f680 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:17.520135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:17.521048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:17.522207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:17.522907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:17.588545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:17.590023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57c008-d86e-4e09-90d9-0a37c8215c50 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:17.600020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1123/2242] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:21.691613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:21.757367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32807d1b-f79c-4229-a265-4cf0042af495 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:21.761718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:21.762589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:21.762869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:21.763152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] PATH_INFO: `/v3/users/9209496c805a46d5b28427ef4b9d5f7f` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:21.764344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9209496c805a46d5b28427ef4b9d5f7f)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:21.766290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:21.816757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9209496c805a46d5b28427ef4b9d5f7f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:21.825043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9209496c805a46d5b28427ef4b9d5f7f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:21.825893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9209496c805a46d5b28427ef4b9d5f7f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:21.839320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9209496c805a46d5b28427ef4b9d5f7f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:21.848318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9209496c805a46d5b28427ef4b9d5f7f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:21.855335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1120/2243] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:21 2022] DELETE /identity/v3/users/9209496c805a46d5b28427ef4b9d5f7f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:21.860632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c09f5b-b724-41bc-b827-61091913563b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:21.861374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c09f5b-b724-41bc-b827-61091913563b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:21.861374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c09f5b-b724-41bc-b827-61091913563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:21.883052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-96c09f5b-b724-41bc-b827-61091913563b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:21.884295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-96c09f5b-b724-41bc-b827-61091913563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:21.888632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:21.890059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:21.922114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:21.961513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1124/2244] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:21.972777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda72ec0-3573-459e-a018-dc5ff6015bcc admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:21.973226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c09f5b-b724-41bc-b827-61091913563b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:21.975121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-96c09f5b-b724-41bc-b827-61091913563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:21.978012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:21.978995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:21.979352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:21.979700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] PATH_INFO: `/v3/users/fce3221c478e4addb801a8581928bbc2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:21.981019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce3221c478e4addb801a8581928bbc2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:21.983350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.045331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c09f5b-b724-41bc-b827-61091913563b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:22.048139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce3221c478e4addb801a8581928bbc2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.049015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:22.049663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:22.049949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:22.050228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:22.057203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce3221c478e4addb801a8581928bbc2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.058877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce3221c478e4addb801a8581928bbc2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.060242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:22.061739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.066831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:22.080505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1125/2245] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:14:22.093918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce3221c478e4addb801a8581928bbc2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.102010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce3221c478e4addb801a8581928bbc2'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.108639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1121/2246] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:21 2022] DELETE /identity/v3/users/fce3221c478e4addb801a8581928bbc2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:22.357920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:22.369840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99253d90-8b03-4de4-812f-5138ba475679 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:22.444946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:22.447204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:22.495996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:22.496960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:22.497270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:22.497702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] PATH_INFO: `/v3/projects/17c8de06d59a4968a103f8f72d6a501f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:22.498964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c8de06d59a4968a103f8f72d6a501f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:22.501344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.543354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95c93-8144-4c41-9371-e09e2a8eaad9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:22.556293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c8de06d59a4968a103f8f72d6a501f', 'request_id': 'req-061d731f-3998-43df-80f7-97b6af4f21cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.573703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c8de06d59a4968a103f8f72d6a501f', 'request_id': 'req-061d731f-3998-43df-80f7-97b6af4f21cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.579348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c8de06d59a4968a103f8f72d6a501f', 'request_id': 'req-061d731f-3998-43df-80f7-97b6af4f21cb'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.603044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] The token cache is being invalidate because project 17c8de06d59a4968a103f8f72d6a501f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:22.603706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.605320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.609612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1126/2247] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:22 2022] DELETE /identity/v3/projects/17c8de06d59a4968a103f8f72d6a501f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:22.614150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:22.616620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:22.617672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:22.642215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:22.643085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:22.643373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:22.643660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] PATH_INFO: `/v3/projects/b08b129b334b4f77b9d0f31e9c970797` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:22.644893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08b129b334b4f77b9d0f31e9c970797)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:22.647484 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.700950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08b129b334b4f77b9d0f31e9c970797', 'request_id': 'req-ab9396be-8a29-446f-a1d5-c565d4061094'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.705072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d731f-3998-43df-80f7-97b6af4f21cb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:22.707557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08b129b334b4f77b9d0f31e9c970797', 'request_id': 'req-ab9396be-8a29-446f-a1d5-c565d4061094'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.715308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08b129b334b4f77b9d0f31e9c970797', 'request_id': 'req-ab9396be-8a29-446f-a1d5-c565d4061094'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.728330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] The token cache is being invalidate because project b08b129b334b4f77b9d0f31e9c970797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:22.728819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.729543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.731236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1122/2248] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:22 2022] DELETE /identity/v3/projects/b08b129b334b4f77b9d0f31e9c970797 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:22.735174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:22.736583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:22.743153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:22.772542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:22.773545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:22.773833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:22.774338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] PATH_INFO: `/v3/users/d5612c703f9e4bac9cf71d46adc1500a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:22.775745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5612c703f9e4bac9cf71d46adc1500a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:22.778095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.811850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9396be-8a29-446f-a1d5-c565d4061094 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:22.820367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:22.821764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:22.822329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:22.822814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] PATH_INFO: `/v3/users/683593da3655461aad4e789e09324fd3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:22.824550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=683593da3655461aad4e789e09324fd3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:22.827611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:22.841728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5612c703f9e4bac9cf71d46adc1500a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.853317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5612c703f9e4bac9cf71d46adc1500a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.855139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5612c703f9e4bac9cf71d46adc1500a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.877762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5612c703f9e4bac9cf71d46adc1500a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.878859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5612c703f9e4bac9cf71d46adc1500a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.901346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683593da3655461aad4e789e09324fd3'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.904189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1127/2249] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:22 2022] DELETE /identity/v3/users/d5612c703f9e4bac9cf71d46adc1500a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:22.918764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683593da3655461aad4e789e09324fd3'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.919930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683593da3655461aad4e789e09324fd3'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.939147 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683593da3655461aad4e789e09324fd3'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.944434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:22.951999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683593da3655461aad4e789e09324fd3'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:22.962553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1123/2250] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:22 2022] DELETE /identity/v3/users/683593da3655461aad4e789e09324fd3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:23.142220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.149409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.243949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39586e-65cc-4b1d-b3e8-44cc56992402 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.295173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.296962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.333553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.334576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.334870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.335147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] PATH_INFO: `/v3/projects/aa4a2a9d25a34e73b8202cece862c68b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.336389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4a2a9d25a34e73b8202cece862c68b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.338396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.345213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:23.381392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6658c1-1830-4ef6-a01f-72c82592dd2d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.395883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4a2a9d25a34e73b8202cece862c68b', 'request_id': 'req-07df5779-16a9-4f38-9fc6-b5c9842aaf03'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.403605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4a2a9d25a34e73b8202cece862c68b', 'request_id': 'req-07df5779-16a9-4f38-9fc6-b5c9842aaf03'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.414325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4a2a9d25a34e73b8202cece862c68b', 'request_id': 'req-07df5779-16a9-4f38-9fc6-b5c9842aaf03'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.426311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] The token cache is being invalidate because project aa4a2a9d25a34e73b8202cece862c68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:23.426626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.427288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-07df5779-16a9-4f38-9fc6-b5c9842aaf03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.429439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1128/2251] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:23 2022] DELETE /identity/v3/projects/aa4a2a9d25a34e73b8202cece862c68b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:23.434019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ca59c-2ce0-42b1-85c3-b8439c46f460 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.434318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ca59c-2ce0-42b1-85c3-b8439c46f460 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.434671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ca59c-2ce0-42b1-85c3-b8439c46f460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.444429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.445320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.445641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.445920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] PATH_INFO: `/v3/projects/c984683e6cc4404fa79fc5101fa5aab1` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.447110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c984683e6cc4404fa79fc5101fa5aab1)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.449096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.452566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-801ca59c-2ce0-42b1-85c3-b8439c46f460 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:23.453790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-801ca59c-2ce0-42b1-85c3-b8439c46f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.462669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1129/2252] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:23.470494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.471205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.471492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.497338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c984683e6cc4404fa79fc5101fa5aab1', 'request_id': 'req-d99a4963-28c9-411c-8239-21c730bbdea1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.498537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:23.500079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.503675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c984683e6cc4404fa79fc5101fa5aab1', 'request_id': 'req-d99a4963-28c9-411c-8239-21c730bbdea1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.512685 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c984683e6cc4404fa79fc5101fa5aab1', 'request_id': 'req-d99a4963-28c9-411c-8239-21c730bbdea1'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.526622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] The token cache is being invalidate because project c984683e6cc4404fa79fc5101fa5aab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:23.526913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.527532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d99a4963-28c9-411c-8239-21c730bbdea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:23.529165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1124/2253] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:23 2022] DELETE /identity/v3/projects/c984683e6cc4404fa79fc5101fa5aab1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:23.533678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1732-557c-4437-b5f4-e72d4ae632e1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.533968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1732-557c-4437-b5f4-e72d4ae632e1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.534249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1732-557c-4437-b5f4-e72d4ae632e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.554083 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6c6b1732-557c-4437-b5f4-e72d4ae632e1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:23.555179 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6c6b1732-557c-4437-b5f4-e72d4ae632e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.577841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1130/2254] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:23.582127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.585426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:23.591682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563ae81-c65d-4c99-a755-69e05289ab4a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.596890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.597545 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.597829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.598108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.598834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:23.599143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.601501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.613551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1131/2255] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:14:23.618637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.621604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1125/2256] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:23.628915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.629208 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.629489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.635014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d7993-b0dc-46ed-9c80-c6caa5415de4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.640035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.640878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.641174 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.641561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.642319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.644244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.647990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:23.649247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:23.657542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1126/2257] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:23.661455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.672749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccf0dd-f21e-49e7-8920-4932df660442 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.677996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1132/2258] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:14:23.679405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.680068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.680342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.680659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.681383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:23.681688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.683279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.691066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.696087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1127/2259] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:23.700386 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.710752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a817dd-655a-4f11-8d28-0c2a2128327e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.751190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:23.763349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae44c-d92a-4c5f-ab33-395180761632 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.769195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.769955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.770241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.770523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.772293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.774270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.776120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.776840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.777126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.777407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.778094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.780006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.798500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1133/2260] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:14:23.799625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.808697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1128/2261] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:14:23.810902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad5029-cc9c-4775-837a-018d17515063 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.811557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.869331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.870014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.870374 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.870656 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.871388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.873125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.874171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a8ef-cb25-4db3-ab33-428f5940beb2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.878851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.879518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.879802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.880081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.880862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.882786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.883946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1134/2262] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:23.888794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.904548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1129/2263] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:14:23.906214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b295723-6bae-446b-bee4-87c4bbef37ca admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.908882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.911472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.912727 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.913020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.913294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.914022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.915657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.919492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ef064-69da-45b8-8d14-dff6be4c033e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.924252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.924938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.925227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.925508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.926048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1135/2264] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:23.926234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.928178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.930072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.940533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1130/2265] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:23.940694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646cbec-abe1-488e-a157-aa2d70a9bd92 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.942337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.944932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.945819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.946102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.946378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] PATH_INFO: `/v3/projects/df7c834822764455941b64d50f506edc/users/15007b467c9245fbb2091c4cf4d6b8c4/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.952948 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660a638-b043-47e3-a2b7-3d3c8fae9f23 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.957628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:23.958282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.958561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.959406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:23.960315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.961963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.962385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7c834822764455941b64d50f506edc, user_id=15007b467c9245fbb2091c4cf4d6b8c4, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:23.965109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96658e2c-016e-4e49-8424-369a4b678da5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:23.975923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1131/2266] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:23.979870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:23.983147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1136/2267] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:23 2022] PUT /identity/v3/projects/df7c834822764455941b64d50f506edc/users/15007b467c9245fbb2091c4cf4d6b8c4/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:23.989647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455755-b9e0-4e7f-b898-b1638022eb67 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:23.993684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:23.993821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:23.994016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:23.994243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:24.020879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] MFA Rules not processed for user `15007b467c9245fbb2091c4cf4d6b8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:24.023284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:24.053931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:24.054849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:24.055161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:24.055421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] PATH_INFO: `/v3/projects/1e8e404c5d674746b33edec6bf8fff2b/users/170f31726faf4737ab36b4072266f8ea/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:24.070165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8e404c5d674746b33edec6bf8fff2b, user_id=170f31726faf4737ab36b4072266f8ea, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:24.073535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80a0ab-dc85-4f5f-b29d-fc49a77963f2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:24.090173 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1132/2268] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:23 2022] PUT /identity/v3/projects/1e8e404c5d674746b33edec6bf8fff2b/users/170f31726faf4737ab36b4072266f8ea/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:24.098190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1137/2269] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:24.106427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:24.106866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:24.107276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:24.110211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:24.111963 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:24.139361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] MFA Rules not processed for user `170f31726faf4737ab36b4072266f8ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:24.140666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:24.160717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:24.203479 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4a647-bca9-40a4-ba8e-b13eedae7bc7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:24.210375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1133/2270] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:24.221646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:24.222981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:24.254316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:24.254967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:24.255248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:24.256184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:24.281293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba3ae-fc74-4c98-a08c-fcfe97c1a149 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:24.284952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:24.285590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:24.285868 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:24.286141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:24.294862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:24.296285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:24.305956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1134/2271] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:24.315407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:24.316920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e643b33-482b-4606-8f52-ecfb095f4f42 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:24.326875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1138/2272] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:14:34.305922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:34.322520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=62 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:34.972739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:35.063818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4eb-499f-41ad-b4c8-2594bb262bf7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.064115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4eb-499f-41ad-b4c8-2594bb262bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.064750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4eb-499f-41ad-b4c8-2594bb262bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.086788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-48f9a4eb-499f-41ad-b4c8-2594bb262bf7 None None] MFA Rules not processed for user `d25dc63e8a4c4af8af2fe29b7819212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:35.088015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-48f9a4eb-499f-41ad-b4c8-2594bb262bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:35.098090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba4502-dcbd-447d-a7b1-e02092358868 service glance] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:35.104835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:35.106983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.107442 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.107906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] PATH_INFO: `/v3/users/15007b467c9245fbb2091c4cf4d6b8c4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.109313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15007b467c9245fbb2091c4cf4d6b8c4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:35.111830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:35.171790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1139/2273] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:35.186188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15007b467c9245fbb2091c4cf4d6b8c4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.197109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15007b467c9245fbb2091c4cf4d6b8c4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.197991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15007b467c9245fbb2091c4cf4d6b8c4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.224942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15007b467c9245fbb2091c4cf4d6b8c4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.235732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15007b467c9245fbb2091c4cf4d6b8c4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.243444 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1135/2274] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:34 2022] DELETE /identity/v3/users/15007b467c9245fbb2091c4cf4d6b8c4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:35.248147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b142-f075-4b55-a372-6d5b74e98b2a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.248588 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b142-f075-4b55-a372-6d5b74e98b2a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.249255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199b142-f075-4b55-a372-6d5b74e98b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.271001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c199b142-f075-4b55-a372-6d5b74e98b2a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:35.272272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c199b142-f075-4b55-a372-6d5b74e98b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:35.339610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1140/2275] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:35.372018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:35.373719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:35.440212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c3537-0f0f-43e0-8e18-24abce5c627e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:35.447562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:35.448534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.448896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.449248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.461798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:35.463358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:35.474718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1136/2276] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:35.639574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee286a-a503-497f-a7e4-af29a4dce5bb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.639876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee286a-a503-497f-a7e4-af29a4dce5bb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.640208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee286a-a503-497f-a7e4-af29a4dce5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.658761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7aee286a-a503-497f-a7e4-af29a4dce5bb None None] MFA Rules not processed for user `ced10e76e5cc4443819157c821f82226`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:35.660156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7aee286a-a503-497f-a7e4-af29a4dce5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:35.722212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1141/2277] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:35.869029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:35.870475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:35.886815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c243f-c044-49b3-9b88-0a7f14dd6907 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:35.891819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:35.892736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.893023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.893306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] PATH_INFO: `/v3/projects/df7c834822764455941b64d50f506edc` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:35.895288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7c834822764455941b64d50f506edc)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:35.897646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:35.916394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:35.951391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7c834822764455941b64d50f506edc', 'request_id': 'req-74b731ec-2c90-4e45-845e-015f9f5abe34'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.957731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7c834822764455941b64d50f506edc', 'request_id': 'req-74b731ec-2c90-4e45-845e-015f9f5abe34'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.963581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7c834822764455941b64d50f506edc', 'request_id': 'req-74b731ec-2c90-4e45-845e-015f9f5abe34'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.977829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] The token cache is being invalidate because project df7c834822764455941b64d50f506edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:35.978049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.978664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-74b731ec-2c90-4e45-845e-015f9f5abe34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:35.980230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1137/2278] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:35 2022] DELETE /identity/v3/projects/df7c834822764455941b64d50f506edc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:35.986659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:35.986953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:35.987234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.007296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:36.009243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:36.055418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:36.072315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1142/2279] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:36.079807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.080101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.080380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.098968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:36.100223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:36.108858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1138/2280] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:36.113338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.123980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b66b2-403b-46e2-99ff-514eaf524fdd None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.128652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.129318 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.129607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.129889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.130629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:36.130943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.132673 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.144559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1143/2281] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:36.148981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.158510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeea8f-6c66-4188-b9fc-5bc9c568f2cc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.163402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.164655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.164933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.165203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.165899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.167818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.195170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1139/2282] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:36 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:36.199689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.210335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05229737-b5e4-41fb-aeec-813455d00e16 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.273977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.274663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.274948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.275224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.275960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.278009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.300348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1144/2283] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:36 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:36.305220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.315392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb3c22-1950-44ae-b374-d0b9077dac4d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.320548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.321223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.321507 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.321783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.322522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.324138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.335895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1140/2284] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:36.340025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.350231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a1014-34cf-401f-8e52-38542c2130ee admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.355436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.356429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.356745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.357048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.357765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.359376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3609c-b54d-4768-a4b6-edbc00dd47da admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.370412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1145/2285] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:36.374499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.388338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d62adc-6d4d-49db-8712-f14119fa8f4e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.392884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.393784 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.394073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.394333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] PATH_INFO: `/v3/projects/b14f7f84a9fa48f0985d0a66332b8f1d/users/be6fbb553cee41319831ee8ef63b932e/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.410209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14f7f84a9fa48f0985d0a66332b8f1d, user_id=be6fbb553cee41319831ee8ef63b932e, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.412917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.428016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1141/2286] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:36 2022] PUT /identity/v3/projects/b14f7f84a9fa48f0985d0a66332b8f1d/users/be6fbb553cee41319831ee8ef63b932e/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:36.439000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5574e-b4ab-4c88-a9d5-3b90d4db8bd7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.439410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5574e-b4ab-4c88-a9d5-3b90d4db8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.439706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5574e-b4ab-4c88-a9d5-3b90d4db8bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.474522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-84b5574e-b4ab-4c88-a9d5-3b90d4db8bd7 None None] MFA Rules not processed for user `be6fbb553cee41319831ee8ef63b932e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:36.475783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-84b5574e-b4ab-4c88-a9d5-3b90d4db8bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:36.547421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1146/2287] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:36.558315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:36.560026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:36.620124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a87b7e-9c8c-4ce9-830e-16b3d6fb51fa admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:36.623922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:36.624659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:36.624946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:36.625227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:36.634485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:36.635921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:36.644672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1142/2288] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:38.002340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a24dc-d1b4-4b78-b313-868a2d3c2d3a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.002849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a24dc-d1b4-4b78-b313-868a2d3c2d3a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.002971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a24dc-d1b4-4b78-b313-868a2d3c2d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.022371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-797a24dc-d1b4-4b78-b313-868a2d3c2d3a None None] MFA Rules not processed for user `be6fbb553cee41319831ee8ef63b932e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:38.024257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-797a24dc-d1b4-4b78-b313-868a2d3c2d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:38.031198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1147/2289] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:46.640714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:47.558174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:47.559684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:47.633043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad26b8-a135-4c41-bc05-45549c3191ab service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:47.637970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:47.638845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.639129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.639406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] PATH_INFO: `/v3/users/170f31726faf4737ab36b4072266f8ea` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.640670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=170f31726faf4737ab36b4072266f8ea)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:47.642620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:47.699795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170f31726faf4737ab36b4072266f8ea'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:47.711172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170f31726faf4737ab36b4072266f8ea'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:47.711915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170f31726faf4737ab36b4072266f8ea'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:47.726930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170f31726faf4737ab36b4072266f8ea'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:47.737494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170f31726faf4737ab36b4072266f8ea'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:47.745744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1143/2290] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:47 2022] DELETE /identity/v3/users/170f31726faf4737ab36b4072266f8ea => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:47.750056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.750534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.750622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.772336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:47.773644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:47.814136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:47.856636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1148/2291] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:47.865822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:47.868858 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:47.961234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556276-a978-47bb-b87b-7e76789abbae admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:47.967346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:47.967944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.968231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.968543 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.978646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:47.980192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4fef-ce80-4fd3-9161-184c7ea0122c service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:47.992002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1144/2292] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:48.366324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.368894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:48.384040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d970e-2d2d-495f-90f4-d1f2b5a6ba35 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.390262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.391272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.391660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.392040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] PATH_INFO: `/v3/projects/1e8e404c5d674746b33edec6bf8fff2b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.394213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8e404c5d674746b33edec6bf8fff2b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.396313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.465493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8e404c5d674746b33edec6bf8fff2b', 'request_id': 'req-852eae1b-3051-4e51-b9db-35477484a8c2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:48.472613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8e404c5d674746b33edec6bf8fff2b', 'request_id': 'req-852eae1b-3051-4e51-b9db-35477484a8c2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:48.486275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8e404c5d674746b33edec6bf8fff2b', 'request_id': 'req-852eae1b-3051-4e51-b9db-35477484a8c2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:48.498212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] The token cache is being invalidate because project 1e8e404c5d674746b33edec6bf8fff2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:48.498510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:48.499168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-852eae1b-3051-4e51-b9db-35477484a8c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:48.500755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1149/2293] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:48 2022] DELETE /identity/v3/projects/1e8e404c5d674746b33edec6bf8fff2b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:48.506510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.506808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.507124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.525564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:48.526902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:48.547939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:48.607924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1145/2294] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:48.609786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.610146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.610531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.631358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:48.633095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:48.641411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1150/2295] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:48.646219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.657048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb517c-8db7-4e0d-b5a5-654c289494ba None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.661981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.662652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.662934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.663216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.664540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:48.664854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.666469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.678439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1146/2296] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:48.682923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.693995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67e34-6412-48e7-9e69-7904192516f4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.698908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.699600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.699912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.700217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.700964 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.702928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.731134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1151/2297] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:48.735265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.746131 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418267f8-ebef-46f3-8676-b69985d28293 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.804040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.804889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.805193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.805481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.806196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.808180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.831262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1147/2298] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:48 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:48.837297 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.852159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9efc0e-c334-465d-a7c7-7888c86d53b1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.857595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.858295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.858581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.858864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.859609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.861560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.872923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1152/2299] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:48.878364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.889860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5191472-004a-40f6-bf1c-9178dad399a2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.894682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.895351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.895666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.895947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.896699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.899025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8f482-7de9-4aab-ab37-e2f3614ac8d3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.909652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1148/2300] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:48.914491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.925806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ca22b-6046-4606-94d2-f57c275a502b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.930512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.931401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.931681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.931953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] PATH_INFO: `/v3/projects/b9fd239910a6492a953310873fd2379e/users/943b21e6079646dc9f47dd5698278b34/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.947431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fd239910a6492a953310873fd2379e, user_id=943b21e6079646dc9f47dd5698278b34, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.950200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:48.970967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1153/2301] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:48 2022] PUT /identity/v3/projects/b9fd239910a6492a953310873fd2379e/users/943b21e6079646dc9f47dd5698278b34/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:48.977976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2d147-eb57-4444-bf7d-9986ff3a4315 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.978265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2d147-eb57-4444-bf7d-9986ff3a4315 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.978549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2d147-eb57-4444-bf7d-9986ff3a4315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.998451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-edf2d147-eb57-4444-bf7d-9986ff3a4315 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:48.999665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-edf2d147-eb57-4444-bf7d-9986ff3a4315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.072279 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1149/2302] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:49.082069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:49.083833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.142915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fc702-ec8f-429b-b313-657fb78ea4fe admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:49.146487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:49.147141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:49.147431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:49.149041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:49.157679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:49.159133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:49.169489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1154/2303] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:54.570455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.571351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.571787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.607296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] MFA Rules not processed for user `943b21e6079646dc9f47dd5698278b34`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:54.608612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.627957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:54.697451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1150/2304] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.700997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.705890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:54.729865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82cc23-0937-485e-87e2-22596a8694cf service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.754435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.755837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.756355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.756816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.758799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.761065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.799988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1155/2305] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:54.804790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.829944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff56683-c2d8-4cc7-9767-75db5217e5ec None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.927754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.928511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.928825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.929274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.929994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.932711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.986918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1151/2306] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:54.990023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.008149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a07f77-1116-4ed7-b503-e57f4f591b67 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.017151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.018542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.019376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.020232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.023621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.026866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.054873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.055415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1156/2307] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:55.076998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614ce0e-f1c3-4d87-9d45-2430c52bfef6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.084670 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.087041 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.087377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.087665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] PATH_INFO: `/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8/users/308da06647e14ea5bf49e915b2777ccc/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.114456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e230c3dbf81400e9bb37fd3d4300bb8, user_id=308da06647e14ea5bf49e915b2777ccc, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.119157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.140495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1152/2308] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:55 2022] PUT /identity/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8/users/308da06647e14ea5bf49e915b2777ccc/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:55.143952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.162212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb03fd4-b611-4530-a7fa-cbbea556aa95 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.261175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.261998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.262525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.262654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.263525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.266181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.287499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1157/2309] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:55.288800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.305456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755eb093-dd12-41d8-a76d-0c86314cb4bf admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.311833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.312820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.313107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.313383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] PATH_INFO: `/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8/users/308da06647e14ea5bf49e915b2777ccc/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.315637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e230c3dbf81400e9bb37fd3d4300bb8, user_id=308da06647e14ea5bf49e915b2777ccc, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.319365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a917e9-c486-4cbe-a5c1-60f17525b457 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.338040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1153/2310] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:55 2022] PUT /identity/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8/users/308da06647e14ea5bf49e915b2777ccc/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:56.098375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:56.102268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.128320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6368787-bdfd-45e0-8220-e35e3bf37d48 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:56.388018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:56.389985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.390200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.390477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] PATH_INFO: `/v3/users/59de120fab9f477f9a48a0b6f2af927a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.400183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59de120fab9f477f9a48a0b6f2af927a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:56.405031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:56.502853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59de120fab9f477f9a48a0b6f2af927a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.517635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59de120fab9f477f9a48a0b6f2af927a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.519208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59de120fab9f477f9a48a0b6f2af927a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.545724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:56.548476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59de120fab9f477f9a48a0b6f2af927a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.550062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59de120fab9f477f9a48a0b6f2af927a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.580320 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1158/2311] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:14:56 2022] DELETE /identity/v3/users/59de120fab9f477f9a48a0b6f2af927a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:56.587628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.588358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.589704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.625643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:56.627822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.760393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1154/2312] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:56.773662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:56.775333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.847967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b06ff-04c5-4721-bb7f-e8bcced6a22c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:56.852823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:56.853549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.853834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.854115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.891464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:56.892994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615ec76-3289-4e11-9a7d-c3aa53ebfcee service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:56.924493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1159/2313] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:58.578262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:58.579903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.599350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75670-31cf-4854-8ed2-eab5410f9cf7 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:58.611308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:58.612219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.612671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.612939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] PATH_INFO: `/v3/projects/ceb79f09ffc544199432d8627b2d1bf4` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.614807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb79f09ffc544199432d8627b2d1bf4)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:58.618896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.632351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:58.735157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb79f09ffc544199432d8627b2d1bf4', 'request_id': 'req-02f1729e-6b09-4afd-bf41-07c5702cad80'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:58.743246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb79f09ffc544199432d8627b2d1bf4', 'request_id': 'req-02f1729e-6b09-4afd-bf41-07c5702cad80'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:58.749957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb79f09ffc544199432d8627b2d1bf4', 'request_id': 'req-02f1729e-6b09-4afd-bf41-07c5702cad80'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:58.778613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] The token cache is being invalidate because project ceb79f09ffc544199432d8627b2d1bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:58.778613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:58.780021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-02f1729e-6b09-4afd-bf41-07c5702cad80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:58.781470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1155/2314] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:14:58 2022] DELETE /identity/v3/projects/ceb79f09ffc544199432d8627b2d1bf4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:58.786389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.790636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.790932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.824242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:58.825522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.953148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1160/2315] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:58.960872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.962008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.962510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.017732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:59.019398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.043456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1156/2316] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:59.047777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.079009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cc780-6ae6-4961-bf4c-ad2a5134bd33 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.127224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.128175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.128473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.128846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.129506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:59.129811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.131520 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.193517 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1161/2317] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:59.195055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.212247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c768ba-5e37-4f0d-90f9-cfd7ea6acde2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.246480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.247177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.247655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.247941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.249081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.251501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.290595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1157/2318] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:14:59 2022] POST /identity/v3/projects => generated 399 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:59.293158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.308657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875968d8-889e-4329-8d77-8061cd9fd1c3 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.396300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.397147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.398316 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.398448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.399277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.401921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.436278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1162/2319] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:14:59 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:59.440471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.457544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0da0-1fcc-4504-8e53-2f2db186c68f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.464294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.465117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.465476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.465824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.466895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.468657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.481407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1158/2320] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:59.485095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.492761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:59.496987 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f930-9bd2-487a-ab41-475b05203057 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.503629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.505145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.506671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.507416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.508106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.510033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6be3cd-c353-464e-8167-0852563da58b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.531373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1163/2321] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:59.535745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.558430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135dc5d8-bded-46d7-bf89-bfe898874e1b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.565794 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.566759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.567050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.567331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] PATH_INFO: `/v3/projects/55f4e41c111444bfbe87e1f7b60dee15/users/df36226405924c81a5bc7d284417a34e/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.594214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f4e41c111444bfbe87e1f7b60dee15, user_id=df36226405924c81a5bc7d284417a34e, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.597582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.628533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1159/2322] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:14:59 2022] PUT /identity/v3/projects/55f4e41c111444bfbe87e1f7b60dee15/users/df36226405924c81a5bc7d284417a34e/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:59.649076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a594a-8086-461b-9aa5-4d01e91a2660 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.649604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a594a-8086-461b-9aa5-4d01e91a2660 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.650472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a594a-8086-461b-9aa5-4d01e91a2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.703456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c52a594a-8086-461b-9aa5-4d01e91a2660 None None] MFA Rules not processed for user `df36226405924c81a5bc7d284417a34e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:59.705763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c52a594a-8086-461b-9aa5-4d01e91a2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.875576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1164/2323] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:59.884966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.889578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.028805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc9810-3d2a-43f9-8046-3733ed746aa3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:00.043673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:00.044349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.044680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.044964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.067168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:00.068821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:00.094262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1160/2324] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:02.092011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.092626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.092766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.143100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.147245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.201160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] MFA Rules not processed for user `308da06647e14ea5bf49e915b2777ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:02.203638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.242032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:02.327474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459161e-4b54-4369-b5be-c296c090e02d service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.333364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.334623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.335581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.336034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.338206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.346350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1165/2325] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:02.355288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.357667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.363529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.364809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad493c6-56e0-4178-ae73-39f63059f22c service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.400836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1161/2326] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:02.511302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:02.514746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff08c2-c410-4636-aaf9-7cef03481956 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.519715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.520766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.521165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.521553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.524315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.617523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.630722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.652815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1166/2327] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:02.756381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0dcb5-9c7f-48c7-a6b9-af71b0bd8827 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.756578 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0dcb5-9c7f-48c7-a6b9-af71b0bd8827 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.757877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0dcb5-9c7f-48c7-a6b9-af71b0bd8827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.835018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f2a0dcb5-9c7f-48c7-a6b9-af71b0bd8827 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:02.837071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f2a0dcb5-9c7f-48c7-a6b9-af71b0bd8827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.992640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1162/2328] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:03.041663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.044188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:03.168948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2837669-d120-401f-aa3c-db0582821fd5 service glance] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.174602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.175393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.175784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.176165 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.187200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.189719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.207716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1167/2329] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:06.766849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3537-0e0d-47d4-aee4-aa6aed0e51e9 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.767379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3537-0e0d-47d4-aee4-aa6aed0e51e9 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.767475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f3537-0e0d-47d4-aee4-aa6aed0e51e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.794034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-658f3537-0e0d-47d4-aee4-aa6aed0e51e9 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:06.795292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-658f3537-0e0d-47d4-aee4-aa6aed0e51e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.806362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1163/2330] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:06.826811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:06.839116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6606f-3224-4f84-982b-c1f6394aacc8 service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:06.847915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:06.848606 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.849061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.849614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.862450 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.864357 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.874319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1168/2331] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:07.603409 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.603698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.604196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.630391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:07.633540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.649664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1164/2332] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:07.662949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.674781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee804fa-491a-43f8-bcdd-f3db0e59196c service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.684564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.685474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.685864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.686230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.700386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.702710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d6e55-6bc5-4ff1-a28d-e9068deaa272 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.715501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1169/2333] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:07.893019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.055038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.074623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.075098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.075200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.090354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057f138-f5c7-4987-8a9c-1de2ce0ee25b None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.099314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.100420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.100730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.101004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] PATH_INFO: `/v3/users/df36226405924c81a5bc7d284417a34e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.102390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df36226405924c81a5bc7d284417a34e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.108612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.115409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.118988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.137500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1170/2334] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.157432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.188008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b689a-43de-449a-ba15-bf3ac401478e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.199191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.199929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.200415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.200675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.212811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.214568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f2e6d-a05d-4bcc-91a2-a0d1bd02804e service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.226431 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1171/2335] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:08.249533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df36226405924c81a5bc7d284417a34e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.271433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df36226405924c81a5bc7d284417a34e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.273184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df36226405924c81a5bc7d284417a34e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.296603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df36226405924c81a5bc7d284417a34e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.312213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df36226405924c81a5bc7d284417a34e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.324869 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1165/2336] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:15:07 2022] DELETE /identity/v3/users/df36226405924c81a5bc7d284417a34e => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:08.329171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc860185-4843-43a0-9a96-5229e74ebb2e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.329480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc860185-4843-43a0-9a96-5229e74ebb2e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.329760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc860185-4843-43a0-9a96-5229e74ebb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.380293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cc860185-4843-43a0-9a96-5229e74ebb2e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.392005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc860185-4843-43a0-9a96-5229e74ebb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.454894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.552638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1172/2337] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.566210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.568071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.718739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685aabac-c775-432a-8fac-118fb99c6822 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.729932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.730622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.730905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.731187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.764097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.765603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.801619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1166/2338] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:08.823271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.823566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.823875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.859380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.860565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.933140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1173/2339] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.950495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.955024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.126010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3938c-5406-4265-b146-556b76b0ddf9 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.142546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.143226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.143506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.143785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.155238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.158120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc8861-48f7-4096-8d02-776f8ee33e43 service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.162249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:09.169528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1167/2340] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:09.358977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.360417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.393986 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c96b6-76e9-4a26-927a-9e65bcdfbf00 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.425155 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.426105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.426414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.426670 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] PATH_INFO: `/v3/projects/55f4e41c111444bfbe87e1f7b60dee15` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.430070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f4e41c111444bfbe87e1f7b60dee15)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.440734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.588887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f4e41c111444bfbe87e1f7b60dee15', 'request_id': 'req-f74ffe03-9ede-4255-9da5-2f6f4465c907'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:09.596957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f4e41c111444bfbe87e1f7b60dee15', 'request_id': 'req-f74ffe03-9ede-4255-9da5-2f6f4465c907'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:09.633932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f4e41c111444bfbe87e1f7b60dee15', 'request_id': 'req-f74ffe03-9ede-4255-9da5-2f6f4465c907'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:09.651158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] The token cache is being invalidate because project 55f4e41c111444bfbe87e1f7b60dee15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:09.652029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:09.654115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f74ffe03-9ede-4255-9da5-2f6f4465c907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:09.656190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1174/2341] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:15:09 2022] DELETE /identity/v3/projects/55f4e41c111444bfbe87e1f7b60dee15 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:09.666861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.667211 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.667501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.697103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:09.698591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.797772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1168/2342] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:09.806755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.809695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.810111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.832635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:09.834025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.848836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.856482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1175/2343] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:09.863843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af46b9-b78a-4566-91e4-b4aee1b612bb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.869519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.870392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.870785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.871165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.872023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:09.872440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.879169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.900157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.907287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1169/2344] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:09.927242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243f172-5231-4c6a-83bf-5f22cfae409b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.937504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.938442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.939075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.939601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.941341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.943783 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.000595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1176/2345] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:10.008229 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.023739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291871df-1c68-43b6-b828-2abd524bdc2e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.109178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.110122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.110596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.111025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.111964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.115289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.151320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1170/2346] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:10.156780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.169249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974aa9ab-ebb0-4618-9c7e-9c4df0b151e1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.174625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.175288 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.175565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.175838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.176613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.180864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.202146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1177/2347] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:10.204496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.224871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf8999-5dd9-43d2-82fb-b4a784195e75 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.236970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.237666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.237938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.238207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.239048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.242334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3614e3-99bc-4474-84ae-a4f41651e50d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.259887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1171/2348] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:10.263407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.278231 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276ca92-01ff-47c9-9b77-6e2b5ee5bab4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.286789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.288871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.289544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.290001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] PATH_INFO: `/v3/projects/32fb068c248a4cb9bb0e73dab923ca19/users/3b7eb6c795964aaa92e7f1fd3993a29c/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.319006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:10.326438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=32fb068c248a4cb9bb0e73dab923ca19, user_id=3b7eb6c795964aaa92e7f1fd3993a29c, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.339444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.364982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1178/2349] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:15:10 2022] PUT /identity/v3/projects/32fb068c248a4cb9bb0e73dab923ca19/users/3b7eb6c795964aaa92e7f1fd3993a29c/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:10.370694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.370694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.370978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.403855 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:10.405104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:10.503122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1172/2350] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:10.509657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.511731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:10.605784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cf9a0-771f-49f1-ae5a-16bbcf32da2a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.612487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.613206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.614535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.614535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.624537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.626013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bcf38-79a0-4959-90ab-85bbdefea5a8 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.644632 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1179/2351] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:11.842184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.843756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.854722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:11.859246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5478589-520d-40f0-9ecf-a3037bc1d972 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.869060 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.870924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.871435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.871943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] PATH_INFO: `/v3/users/be6fbb553cee41319831ee8ef63b932e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.873704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6fbb553cee41319831ee8ef63b932e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.876026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.976502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6fbb553cee41319831ee8ef63b932e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.994285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6fbb553cee41319831ee8ef63b932e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.995045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6fbb553cee41319831ee8ef63b932e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.023733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6fbb553cee41319831ee8ef63b932e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.040068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6fbb553cee41319831ee8ef63b932e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.055927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1173/2352] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:15:11 2022] DELETE /identity/v3/users/be6fbb553cee41319831ee8ef63b932e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:12.057854 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.058352 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.058433 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.081786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:12.082698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.153147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1180/2353] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:12.165540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.166988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.285555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2e1df-1d90-44fc-ad99-8b75a2beb1b6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.302336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.304976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.305497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.306501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.336426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.338352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1e0f-ac31-46ba-bae0-902e37545063 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.351288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1174/2354] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:12.617184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.619405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.635600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197c5a6-f6fd-4880-8dbe-59f6e3f162c1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.646845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.648616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.649007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.649389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] PATH_INFO: `/v3/projects/b14f7f84a9fa48f0985d0a66332b8f1d` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.651111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14f7f84a9fa48f0985d0a66332b8f1d)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.653406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.735790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14f7f84a9fa48f0985d0a66332b8f1d', 'request_id': 'req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.750105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14f7f84a9fa48f0985d0a66332b8f1d', 'request_id': 'req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.788336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14f7f84a9fa48f0985d0a66332b8f1d', 'request_id': 'req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.806561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] The token cache is being invalidate because project b14f7f84a9fa48f0985d0a66332b8f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:12.807677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.809928 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-56c8b3a5-ba6f-4f10-aaca-851d093ff8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.825153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1181/2355] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:15:12 2022] DELETE /identity/v3/projects/b14f7f84a9fa48f0985d0a66332b8f1d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:16.098894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:16.099422 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:16.099552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:16.119531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] MFA Rules not processed for user `943b21e6079646dc9f47dd5698278b34`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:16.120808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:16.203417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1175/2356] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:17.068699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.069439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.069439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.126718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] MFA Rules not processed for user `3b7eb6c795964aaa92e7f1fd3993a29c`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:17.128106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.138330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:17.246210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1182/2357] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:17.250561 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.251966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.286629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:17.342395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a292d5a-0d59-428b-91f4-665c5cb4de61 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.350082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.351817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.351817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.352007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.352729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.354681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.399025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1176/2358] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:15:17 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:17.402980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.419496 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5234c7c-61bb-4acc-ab7e-26fb65724fb3 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.497278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.497952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.498236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.498515 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.499220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.501453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.524488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1183/2359] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:15:17 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:17.527347 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.538845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69b268-0e78-473b-95f3-ddb35e35ac49 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.544159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.544901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.545241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.545557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.546281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.547990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.560140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1177/2360] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:17.564221 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.576481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbecf3-438f-4ddd-9891-954a14eac6d2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.582122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.582979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.583362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.583735 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] PATH_INFO: `/v3/projects/a119c636d156473fb72d5fe4ec32d4da/users/82715aba02d6401b8789590e44292487/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.601500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a119c636d156473fb72d5fe4ec32d4da, user_id=82715aba02d6401b8789590e44292487, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.605040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.619923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1184/2361] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:15:17 2022] PUT /identity/v3/projects/a119c636d156473fb72d5fe4ec32d4da/users/82715aba02d6401b8789590e44292487/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:17.624082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.635986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533d2cc-1757-433d-aa8e-9df09f6e2756 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.699535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.700214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.700540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.700829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.701583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.703232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf535b6-c978-4f7f-a4b0-e2c462f35cf9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.714691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1178/2362] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:17.719112 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.731675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc295bb-e989-4e3f-8573-489c7cc91f42 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.737451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.738451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.738802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.739150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] PATH_INFO: `/v3/projects/a119c636d156473fb72d5fe4ec32d4da/users/82715aba02d6401b8789590e44292487/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.741295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a119c636d156473fb72d5fe4ec32d4da, user_id=82715aba02d6401b8789590e44292487, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.744222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.755252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1185/2363] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:15:17 2022] PUT /identity/v3/projects/a119c636d156473fb72d5fe4ec32d4da/users/82715aba02d6401b8789590e44292487/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:18.511873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404008-13ba-486d-9a48-bca3ff3dc5c0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.512376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404008-13ba-486d-9a48-bca3ff3dc5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.512522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404008-13ba-486d-9a48-bca3ff3dc5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.534061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-77404008-13ba-486d-9a48-bca3ff3dc5c0 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:18.535294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-77404008-13ba-486d-9a48-bca3ff3dc5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.608734 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1179/2364] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:18.624434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:18.626886 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.712140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:18.723294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fdbf8-c5e6-430c-8481-e97b3fd170b2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:18.732531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dbace-a613-4974-b249-1152d866c066 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:18.733573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dbace-a613-4974-b249-1152d866c066 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.733849 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dbace-a613-4974-b249-1152d866c066 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.734122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dbace-a613-4974-b249-1152d866c066 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.743718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dbace-a613-4974-b249-1152d866c066 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:18.745370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dbace-a613-4974-b249-1152d866c066 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:18.759317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1186/2365] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:15:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:15:27.092215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc28f1-6b2d-4a82-bcc2-501df85aefec None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.092789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc28f1-6b2d-4a82-bcc2-501df85aefec None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.092789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc28f1-6b2d-4a82-bcc2-501df85aefec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.126765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c5fc28f1-6b2d-4a82-bcc2-501df85aefec None None] MFA Rules not processed for user `82715aba02d6401b8789590e44292487`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:27.128174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c5fc28f1-6b2d-4a82-bcc2-501df85aefec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.165907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:27.293778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1180/2366] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:27.295564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dbace-a613-4974-b249-1152d866c066 service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.297058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-514dbace-a613-4974-b249-1152d866c066 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.466687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dbace-a613-4974-b249-1152d866c066 service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.474058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.474790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.475786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.476436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.664054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.667217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.679731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1187/2367] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:29.125956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59d60-dd74-4299-ac66-85bcdd81f032 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.126540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59d60-dd74-4299-ac66-85bcdd81f032 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.126838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b59d60-dd74-4299-ac66-85bcdd81f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.147710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a4b59d60-dd74-4299-ac66-85bcdd81f032 None None] MFA Rules not processed for user `3b7eb6c795964aaa92e7f1fd3993a29c`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:29.149268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a4b59d60-dd74-4299-ac66-85bcdd81f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:29.159284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1181/2368] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:37.673806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=70 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:39.141553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=19 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:47.010301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.011752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.070954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb19b8-10a2-4821-a4ae-e55de5dac642 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.074802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.075521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.075827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.076083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.077865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.152736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.154276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.164552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1188/2369] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:57.159605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=50 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:00.879918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cc2fc-996f-48c9-b912-61ddd891f34e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:00.880473 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cc2fc-996f-48c9-b912-61ddd891f34e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:00.880649 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cc2fc-996f-48c9-b912-61ddd891f34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:00.899309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-282cc2fc-996f-48c9-b912-61ddd891f34e None None] MFA Rules not processed for user `ced10e76e5cc4443819157c821f82226`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:00.900554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-282cc2fc-996f-48c9-b912-61ddd891f34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:00.954611 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1182/2370] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:08.741899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:08.743359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:08.754273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b72aec-639d-405d-891c-b95c2b796d0e service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:08.758774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:08.759420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:08.759713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:08.759993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:08.769058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:08.770506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:08.780082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1189/2371] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:10.947969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=25 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:15.721650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.722267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.722267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.739943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] MFA Rules not processed for user `d25dc63e8a4c4af8af2fe29b7819212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:15.741304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:15.796753 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1183/2372] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:18.775905 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:25.789969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=25 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:59.479894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:59.489197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92556e1-196e-4e84-a5db-a28931eb173a service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:59.492850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:59.493484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:59.493769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:59.494050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:59.495811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:59.509333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:59.510920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12c43a-354e-42be-82c4-62e7cebb0e30 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:59.522035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1190/2373] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:16:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:01.514070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:01.515446 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:01.574657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6db5a-099a-49af-b537-a9a6b5192bab None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:01.578302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:01.578970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:01.579254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:01.579524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:01.581297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:01.594808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:01.596220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:01.607856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1184/2374] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:06.122763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:06.123244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:06.123362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:06.141182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] MFA Rules not processed for user `097084a3e57540d18bae448da3b3ecc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:06.142414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:06.201136 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1191/2375] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:06.816437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:06.818105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:06.894226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994139bd-623b-4db7-9e04-952c4b79331e service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:06.899613 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:06.900257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:06.900574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:06.900860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:06.902618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:06.914390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:06.915962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0095802-56eb-4c08-b99c-febafeb4eaa2 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:06.926563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1185/2376] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:12.526085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:12.527493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:12.543844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed93a24-7b43-4de9-933c-36180cf7e514 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:12.549194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:12.549828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:12.550113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:12.550554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:12.552108 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:12.562151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:12.563739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:12.572561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1192/2377] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:16.921558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=58 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:18.066901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.067799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.067799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.085290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] MFA Rules not processed for user `ced10e76e5cc4443819157c821f82226`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:18.086514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:18.093786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1186/2378] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:17:18.417809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.428937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89dae2-e90f-4a8e-8b06-db410eefa72f service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.434375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.435327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.435758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.436286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.438831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:18.451128 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.453869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.463602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1193/2379] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:18.882137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.895901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a69bc-c90b-4544-8d0f-ae36a6ae569d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.905314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.905982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.906319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.906604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.908397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:18.920114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.921580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.930654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1187/2380] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:22.383889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:22.412680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320ff49-c370-45b6-a5e7-3100959de783 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:22.418388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:22.419319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:22.419770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:22.420057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:22.421980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:22.438293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:22.444079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:22.457117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1194/2381] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:22.745188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:22.761381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80d3bf-49a2-43db-8fa2-0dc9907b0a1b service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:22.768156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:22.768844 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:22.769867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:22.770307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:22.772503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:22.789916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:22.792423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1a5d3-25cf-406d-8661-aa228ef6313e service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:22.807910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1188/2382] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:23.712918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:23.723300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9030e9a0-16c7-4621-8b7d-7fecc413ae26 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:23.726962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:23.727587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:23.727868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:23.728194 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:23.730017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:23.741409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:23.742861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:23.753211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1195/2383] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:32.800907 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=15 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:33.754321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=55 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:33.765032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:33.765325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:33.765608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:33.783983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:33.786362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:33.793945 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1189/2384] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:17:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:33.838225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:33.850615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa37b0-680c-450d-9d69-6fd02bcb8582 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:33.855698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:33.856415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:33.856796 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:33.857154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:33.865674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:33.867273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:33.875733 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1196/2385] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:17:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:17:38.198080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:38.199531 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:38.214789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d27d0-b080-4e95-841c-f93ea65e29ff None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:38.219993 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:38.220681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:38.220967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:38.221244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:38.223765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:38.235652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:38.239381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:38.254491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1190/2386] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:40.812768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:40.834086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c089b-92b2-495b-9c0a-a405e9ecef29 service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:40.844543 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:40.845204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.845485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.845785 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.848621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:40.865188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:40.868398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:40.893393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1197/2387] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:41.632829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:41.651673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacb509-b709-49dd-8804-58b1e88817bb service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:41.676047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:41.679292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:41.679589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:41.682913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:41.687035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:41.706714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:41.708169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:41.715629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:41.720827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:41.753966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1191/2388] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:41.807256 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff7961-624c-454b-ac7c-8e39359fa092 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:41.811757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:41.812662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:41.812944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:41.813522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] PATH_INFO: `/v3/users/943b21e6079646dc9f47dd5698278b34` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:41.814400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=943b21e6079646dc9f47dd5698278b34)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:41.817516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:41.868544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943b21e6079646dc9f47dd5698278b34'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:41.879724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943b21e6079646dc9f47dd5698278b34'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:41.880500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943b21e6079646dc9f47dd5698278b34'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:41.898713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943b21e6079646dc9f47dd5698278b34'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:41.899578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943b21e6079646dc9f47dd5698278b34'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:41.914038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1198/2389] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:17:41 2022] DELETE /identity/v3/users/943b21e6079646dc9f47dd5698278b34 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:17:42.093479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:42.106055 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7698b6-6b76-4e61-b5c2-d86e7023bbd8 service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:42.165063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:42.165923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:42.166204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:42.166476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] PATH_INFO: `/v3/projects/b9fd239910a6492a953310873fd2379e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:42.167694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fd239910a6492a953310873fd2379e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:42.169570 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:42.216205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fd239910a6492a953310873fd2379e', 'request_id': 'req-6aca58cc-a57f-43e6-99af-8f9408b20d6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.222635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fd239910a6492a953310873fd2379e', 'request_id': 'req-6aca58cc-a57f-43e6-99af-8f9408b20d6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.228260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fd239910a6492a953310873fd2379e', 'request_id': 'req-6aca58cc-a57f-43e6-99af-8f9408b20d6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.241352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] The token cache is being invalidate because project b9fd239910a6492a953310873fd2379e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:42.241644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.242272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.243809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1192/2390] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:17:42 2022] DELETE /identity/v3/projects/b9fd239910a6492a953310873fd2379e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:42.247835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:42.249295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:42.262997 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7757742-6c04-4b2a-9a35-ebcb1ea9b92c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:42.311492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:42.323215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:42.324105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:42.324417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:42.324726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] PATH_INFO: `/v3/users/308da06647e14ea5bf49e915b2777ccc` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:42.325938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=308da06647e14ea5bf49e915b2777ccc)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:42.327923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:42.377416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308da06647e14ea5bf49e915b2777ccc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.381930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308da06647e14ea5bf49e915b2777ccc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.382567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308da06647e14ea5bf49e915b2777ccc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.395667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308da06647e14ea5bf49e915b2777ccc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.396329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308da06647e14ea5bf49e915b2777ccc'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.408730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1199/2391] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:17:42 2022] DELETE /identity/v3/users/308da06647e14ea5bf49e915b2777ccc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:42.585265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:42.586935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:42.601898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca58cc-a57f-43e6-99af-8f9408b20d6d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:42.655183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:42.663008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:42.663913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:42.664403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:42.664880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] PATH_INFO: `/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:42.666195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e230c3dbf81400e9bb37fd3d4300bb8)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:42.668117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:42.715183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e230c3dbf81400e9bb37fd3d4300bb8', 'request_id': 'req-5b1da359-678c-41cf-88b0-94958831925c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.721601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e230c3dbf81400e9bb37fd3d4300bb8', 'request_id': 'req-5b1da359-678c-41cf-88b0-94958831925c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.727927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e230c3dbf81400e9bb37fd3d4300bb8', 'request_id': 'req-5b1da359-678c-41cf-88b0-94958831925c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.740919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] The token cache is being invalidate because project 1e230c3dbf81400e9bb37fd3d4300bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:42.741206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.741828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:42.743395 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1193/2392] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:17:42 2022] DELETE /identity/v3/projects/1e230c3dbf81400e9bb37fd3d4300bb8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:45.766665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:45.768260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:45.826959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969aa95-8951-457a-939f-bc24568cedb5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:45.877491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:45.878903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:45.879283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:45.879561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:45.881349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:45.923229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:45.945109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:45.946551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:45.956066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1200/2393] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:49.072562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.074673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:49.087661 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da359-678c-41cf-88b0-94958831925c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.092309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.093448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.093732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.094005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] PATH_INFO: `/v3/users/d25dc63e8a4c4af8af2fe29b7819212b` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.095169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25dc63e8a4c4af8af2fe29b7819212b)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.098132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.145582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25dc63e8a4c4af8af2fe29b7819212b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.153920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25dc63e8a4c4af8af2fe29b7819212b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.154612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25dc63e8a4c4af8af2fe29b7819212b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.167692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25dc63e8a4c4af8af2fe29b7819212b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.176263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-2edebe84-c208-4f4c-a6d4-f15663530daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25dc63e8a4c4af8af2fe29b7819212b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.182496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1194/2394] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:17:49 2022] DELETE /identity/v3/users/d25dc63e8a4c4af8af2fe29b7819212b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:49.371011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.388719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72002094-ace2-45d0-a10b-81f199169588 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.447453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.448315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.448640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.448909 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] PATH_INFO: `/v3/projects/c162124048a34243867e236c81a70eb0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.450079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c162124048a34243867e236c81a70eb0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.452029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.497009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c162124048a34243867e236c81a70eb0', 'request_id': 'req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.503048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c162124048a34243867e236c81a70eb0', 'request_id': 'req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.510332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c162124048a34243867e236c81a70eb0', 'request_id': 'req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.521466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] The token cache is being invalidate because project c162124048a34243867e236c81a70eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:49.521697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.522324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-39ba3ba2-1e57-4055-bff8-8fd825f7ac4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:49.523851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1201/2395] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:17:49 2022] DELETE /identity/v3/projects/c162124048a34243867e236c81a70eb0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:49.528861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.529154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.529436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.547915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:49.549156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:49.601669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:49.611477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1195/2396] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:49.619301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.619619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.619904 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.638942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:49.640130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:49.648804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1202/2397] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:49.652970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.662519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56af10-5890-4a74-8621-d0c782f491f1 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.675501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.676148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.676432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.676743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.677461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:17:49.677771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.679377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.691978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1196/2398] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:17:49.696244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.706596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad45bcc-919f-49d0-b532-9530715e5ce1 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.711332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.711984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.712267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.712579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.713275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.715267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.741497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1203/2399] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:17:49 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:17:49.745711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.756240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e639e-e77c-4d19-8ece-eed60094fb2a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.815866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.816704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.816992 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.817294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.817999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.819932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.840176 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1197/2400] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:17:49 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:17:49.844598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.855011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280bec1-b9ff-43f5-ba89-ce1497498053 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.859667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.860324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.860712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.860996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.861715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.863530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.873835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1204/2401] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:49.878930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.888323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc156c66-3d8a-46a1-8ca0-9444da520324 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.892931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.894165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.894445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.894721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.895433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.897072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8274f-f03f-4166-8082-83038fa850c1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.906825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1198/2402] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:49.911276 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:49.921302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be19e53-90cd-4e31-98f6-5eb65f7b6cf7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:49.926224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:49.927106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.927384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.927661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] PATH_INFO: `/v3/projects/80d48e2adc1e4c6ebfdd43e84f8e3106/users/973c068eff44436785ddf19a6465da8f/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.943494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d48e2adc1e4c6ebfdd43e84f8e3106, user_id=973c068eff44436785ddf19a6465da8f, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:49.946222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:49.959726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1205/2403] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:17:49 2022] PUT /identity/v3/projects/80d48e2adc1e4c6ebfdd43e84f8e3106/users/973c068eff44436785ddf19a6465da8f/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:17:49.964706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6ba4-7c51-4944-8170-3cdb0eafec53 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:49.965001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6ba4-7c51-4944-8170-3cdb0eafec53 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:49.965280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6ba4-7c51-4944-8170-3cdb0eafec53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:49.982893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b02a6ba4-7c51-4944-8170-3cdb0eafec53 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:49.984075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b02a6ba4-7c51-4944-8170-3cdb0eafec53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:50.046640 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1199/2404] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:50.056975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:50.058687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:50.066180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:50.117866 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff143fcc-e445-4b95-b638-1e2d4c06f318 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:50.121566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:50.122201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:50.122492 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:50.122767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:50.132553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:50.133998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:50.143603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1206/2405] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:53.950428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e4f-eb0c-4a74-8a51-0879cef7a473 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:53.950979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e4f-eb0c-4a74-8a51-0879cef7a473 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:53.950979 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21e4f-eb0c-4a74-8a51-0879cef7a473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:53.982242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-68a21e4f-eb0c-4a74-8a51-0879cef7a473 None None] MFA Rules not processed for user `973c068eff44436785ddf19a6465da8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:53.983608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-68a21e4f-eb0c-4a74-8a51-0879cef7a473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:54.043006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:54.054977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1200/2406] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:54.071415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:54.072956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:54.152120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78d05b-83d1-4dad-ba0c-94c2404a6b39 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:54.159885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:54.161163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:54.161445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:54.161745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:54.170552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:54.172167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:54.183454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1207/2407] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:55.820284 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:55.820829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:55.821180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:55.841666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] MFA Rules not processed for user `973c068eff44436785ddf19a6465da8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:55.843262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:55.850864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1201/2408] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:17:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:04.179368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=58 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:04.296113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:04.308919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98353624-8714-4318-af0e-59c0978bfd0f service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:04.314621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:04.315567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:04.316046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:04.316336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:04.318197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:04.331907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:04.333622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dab8e-e341-4053-b2ca-b786fdfdcf82 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:04.345491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1208/2409] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:05.835204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:14.339885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:30.164712 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:30.176210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef3ca-2235-4678-8181-dff4cec7cd88 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:30.181009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:30.181710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.182081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.182572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.184506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:30.195910 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:30.197429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d6e38-5f46-4ed0-86e3-e8d715c0fabe service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:30.206895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1202/2410] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:30.321072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.321607 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.321725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.323578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1209/2411] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:18:30.330349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dee768-604a-4dec-97f6-23efa0ddd059 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.330939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dee768-604a-4dec-97f6-23efa0ddd059 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.331441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dee768-604a-4dec-97f6-23efa0ddd059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.352591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-64dee768-604a-4dec-97f6-23efa0ddd059 None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:30.353542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-64dee768-604a-4dec-97f6-23efa0ddd059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:30.360797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1203/2412] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:30.373532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:30.378152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:30.400952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff0e19-5d15-41d1-9a18-438ab682835e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:30.404537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:30.405183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.405469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.405749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.414797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:30.416270 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f8bbb-6379-4676-a787-660423e89ef6 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:30.432828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1210/2413] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:30.528781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a89f87-72f8-41ee-86c2-bfada3a2eccb None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.529075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a89f87-72f8-41ee-86c2-bfada3a2eccb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.529357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a89f87-72f8-41ee-86c2-bfada3a2eccb None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.530766 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1204/2414] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:18:30.539350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.539643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.539929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.581446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:30.582797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:30.593220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1211/2415] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:30.605733 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a89f87-72f8-41ee-86c2-bfada3a2eccb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:30.621031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a89f87-72f8-41ee-86c2-bfada3a2eccb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:30.626401 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:30.628618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:30.628906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:30.629185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:30.643659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:30.646278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:30.669778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1205/2416] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:31.643680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:31.654863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8199aa6-7d02-4602-821d-0b0333300481 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:31.660245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:31.661057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:31.661409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:31.661756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:31.663713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:31.677030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:31.678593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:31.689090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1212/2417] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:32.429359 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:32.446304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c31f19-344a-449a-a503-fe2d427a9ea4 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:32.450133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:32.451049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:32.451416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:32.451756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:32.456049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:32.471887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:32.473535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:32.486420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1206/2418] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:36.149001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:36.152236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:36.167621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1b717-ad7f-4459-827b-63f7b44f0ba5 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:36.173049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:36.174292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.174969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.175253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] PATH_INFO: `/v3/users/097084a3e57540d18bae448da3b3ecc4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.176536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097084a3e57540d18bae448da3b3ecc4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:36.178638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:36.250315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097084a3e57540d18bae448da3b3ecc4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.261477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097084a3e57540d18bae448da3b3ecc4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.262532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097084a3e57540d18bae448da3b3ecc4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.297322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097084a3e57540d18bae448da3b3ecc4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.298646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097084a3e57540d18bae448da3b3ecc4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.331030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1213/2419] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:36 2022] DELETE /identity/v3/users/097084a3e57540d18bae448da3b3ecc4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:36.636783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:36.637340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:36.638242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:36.638830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:36.659786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf0060-6d77-465d-a221-6b9f6fdd5c94 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:36.723307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59317c1d-d16f-47a0-a0df-873bed3dec02 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:36.727309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:36.728029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:36.728240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.728556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.728866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] PATH_INFO: `/v3/users/ced10e76e5cc4443819157c821f82226` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.728992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.729190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.729468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] PATH_INFO: `/v3/projects/72d5173b32b5480790babb3d178b94fb` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.730068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced10e76e5cc4443819157c821f82226)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:36.730655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d5173b32b5480790babb3d178b94fb)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:36.732014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:36.732620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:36.764995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:36.786017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d5173b32b5480790babb3d178b94fb', 'request_id': 'req-0d79dafa-8bde-483b-b3e6-f45f50f6357e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.786974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced10e76e5cc4443819157c821f82226'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.792395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d5173b32b5480790babb3d178b94fb', 'request_id': 'req-0d79dafa-8bde-483b-b3e6-f45f50f6357e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.794070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced10e76e5cc4443819157c821f82226'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.794751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced10e76e5cc4443819157c821f82226'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.801959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d5173b32b5480790babb3d178b94fb', 'request_id': 'req-0d79dafa-8bde-483b-b3e6-f45f50f6357e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.807737 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced10e76e5cc4443819157c821f82226'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.813272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] The token cache is being invalidate because project 72d5173b32b5480790babb3d178b94fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:36.813767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.815351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0d79dafa-8bde-483b-b3e6-f45f50f6357e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.817169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1214/2420] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:36 2022] DELETE /identity/v3/projects/72d5173b32b5480790babb3d178b94fb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:36.820066 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d359bf7c-6c96-49e9-899a-a743a94df883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced10e76e5cc4443819157c821f82226'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:36.822042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.822327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.822616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.826781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1207/2421] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:36 2022] DELETE /identity/v3/users/ced10e76e5cc4443819157c821f82226 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:36.843901 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:36.845151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:36.920723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1215/2422] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:36.920989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.921390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.921751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.942944 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:36.944278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:36.952820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1208/2423] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:36.957278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:36.973811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234ab98-b2ce-40ee-a409-360ea7836828 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:36.981561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:36.982248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:36.982535 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:36.982816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:36.983561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:36.983875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:36.985569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.000503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1216/2424] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:18:37.003103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.017333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685ec29-5dfb-46e0-8396-7712e273a0d2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.022423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.023281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.023510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.023797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.026655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.026655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.029196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.030703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.046857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f80888-6707-4595-95d0-d825bd031cfe admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.053891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.054871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.055149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.056248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] PATH_INFO: `/v3/projects/cc1a70bd2ce64fbcb738a2cfcfb09f41` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.057530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1a70bd2ce64fbcb738a2cfcfb09f41)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.060526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.061916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.082148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1209/2425] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:37.083378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.127289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1a70bd2ce64fbcb738a2cfcfb09f41', 'request_id': 'req-2b05410c-92ff-4895-a14d-4904d8383447'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:37.134148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1a70bd2ce64fbcb738a2cfcfb09f41', 'request_id': 'req-2b05410c-92ff-4895-a14d-4904d8383447'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:37.142219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1a70bd2ce64fbcb738a2cfcfb09f41', 'request_id': 'req-2b05410c-92ff-4895-a14d-4904d8383447'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:37.153303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85382-d71b-4ef8-9a6b-5d05aa59d728 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.153992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] The token cache is being invalidate because project cc1a70bd2ce64fbcb738a2cfcfb09f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:37.154504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:37.154972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2b05410c-92ff-4895-a14d-4904d8383447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:37.157465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1217/2426] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:37 2022] DELETE /identity/v3/projects/cc1a70bd2ce64fbcb738a2cfcfb09f41 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:37.164059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7125e16-2081-47a4-a957-faabeeb73d2b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.164405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7125e16-2081-47a4-a957-faabeeb73d2b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.164689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7125e16-2081-47a4-a957-faabeeb73d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.183832 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c7125e16-2081-47a4-a957-faabeeb73d2b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:37.185080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c7125e16-2081-47a4-a957-faabeeb73d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.207017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.218826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.219562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.219863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.220250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.222155 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.224371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.249724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1210/2427] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:37.253765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.255521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.266457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1218/2428] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:37.268717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301324c7-ba3a-415f-8d55-345beb75773d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.273924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.274774 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.275383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.275604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.275715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.275874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.276201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.276966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.278689 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.290520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1211/2429] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:37.294096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.295445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:37.296770 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.304018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430cf38-da5f-4934-be8b-efe025d8e998 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.305353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1219/2430] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:37.309266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.309641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.309986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.310272 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.310553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.311326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.313203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.320154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1970d-ed53-4ac1-891c-3b62c70f2924 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.324743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1212/2431] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:37.328013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.328384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.328874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.329232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.330972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.330972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:37.331195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.333002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.340481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ff62-4cd3-4e3e-9e06-f1e6964c8593 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.349861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1220/2432] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:37.350310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.350310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.350310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.350310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] PATH_INFO: `/v3/projects/5c31441c29c54f7789ccd461adc47d2f/users/7e2ca3a5f8ea4cf48421ae85db2999a5/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.351093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.363809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d47e8-67b8-4a41-8ea5-5a14d6896783 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.364361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c31441c29c54f7789ccd461adc47d2f, user_id=7e2ca3a5f8ea4cf48421ae85db2999a5, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.367250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b910e-8159-4ec4-a184-5bfe7e0c7aee admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.368868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.370023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.370383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.370575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.371300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.373575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.384872 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1213/2433] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:37 2022] PUT /identity/v3/projects/5c31441c29c54f7789ccd461adc47d2f/users/7e2ca3a5f8ea4cf48421ae85db2999a5/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:37.396628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.396950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.397286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.412120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1221/2434] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:37.413444 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.427438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] MFA Rules not processed for user `7e2ca3a5f8ea4cf48421ae85db2999a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:37.428743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.465410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.479035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351242e-0ce8-46d4-9b9a-d5866b3665a0 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.483769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.484457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.484752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.485034 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.485788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.488082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.498988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1214/2435] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:37.503970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.510149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1222/2436] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:37.514064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.514994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a1737-080c-4146-8e26-7905c61f5ac8 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.520258 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.520970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.521253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.521529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.522238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.524130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.525042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5031c6b-6b50-45ac-a6bb-c7e21ae31384 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.529961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.530663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.530955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.531336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.532093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.534465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.545542 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1223/2437] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:37.549296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.552330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1215/2438] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:37.556257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.559415 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21637bde-22c7-4035-9e80-676ac40247bc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.619066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.619729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.620014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.620292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.620506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399c37e-04ef-4760-8389-d764315c0a0e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.621045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.622776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.625335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.626005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.626288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.626566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.627798 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.628681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.629755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.634489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1224/2439] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:37.638519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.649399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba3b0-851b-4556-8177-956876c58f5c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.653936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1216/2440] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:37.654523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.654881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.655130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.655407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] PATH_INFO: `/v3/projects/f522ae7a4fb74ca79ba67ced34d92c84/users/110ca5f109264b64a79ad38aaa93e7f7/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.657696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.668627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a9ba6-8a2c-4ff2-9dcb-c10387052338 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.673353 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.673521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f522ae7a4fb74ca79ba67ced34d92c84, user_id=110ca5f109264b64a79ad38aaa93e7f7, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.674164 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.674453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.674736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.675467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.676391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5713a4-c5aa-4963-80bd-0aeb9ca4a447 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.677266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.688871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1217/2441] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:37.690322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1225/2442] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:37 2022] PUT /identity/v3/projects/f522ae7a4fb74ca79ba67ced34d92c84/users/110ca5f109264b64a79ad38aaa93e7f7/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:37.693588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.699983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.700404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.700843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.703841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe1c2b-094e-4196-9b2b-94ab567e1f9e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.729472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] MFA Rules not processed for user `110ca5f109264b64a79ad38aaa93e7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:37.730679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.764010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.764929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.765213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.765492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] PATH_INFO: `/v3/projects/d449019765dc48dc90ed127d5171b91a/users/8a0c4cbed69f4f67bc2573f51983ab3c/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.780698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d449019765dc48dc90ed127d5171b91a, user_id=8a0c4cbed69f4f67bc2573f51983ab3c, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.783609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.797416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1218/2443] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:37 2022] PUT /identity/v3/projects/d449019765dc48dc90ed127d5171b91a/users/8a0c4cbed69f4f67bc2573f51983ab3c/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:37.800498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1226/2444] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:37.801877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.809475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.810953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.820379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.865847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f02923-56f2-439c-a35d-b0b9e53c17ec admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.870515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.871177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.871470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.871746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.872490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.874108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.887848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1219/2445] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:37.888744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:37.890702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d008b0-108c-4273-94cb-f2b801d74ca7 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.899044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a668048-7f07-4c72-a2d0-875cf0d6aa16 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:37.904104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.905157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.905435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.905708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] PATH_INFO: `/v3/projects/d449019765dc48dc90ed127d5171b91a/users/8a0c4cbed69f4f67bc2573f51983ab3c/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.907832 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d449019765dc48dc90ed127d5171b91a, user_id=8a0c4cbed69f4f67bc2573f51983ab3c, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:37.910620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b099f-238e-43a0-8701-58d5b2153fd0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:37.921061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1220/2446] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:37 2022] PUT /identity/v3/projects/d449019765dc48dc90ed127d5171b91a/users/8a0c4cbed69f4f67bc2573f51983ab3c/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:37.931398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.931687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.932180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:37.950967 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:37.961013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] MFA Rules not processed for user `8a0c4cbed69f4f67bc2573f51983ab3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:37.962051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:37.962198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:37.962775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:37.963084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:37.963401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.022445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.024120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:38.033405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1227/2447] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:38.036495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1221/2448] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:38.045571 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:38.046945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:38.110346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb2fa9-7028-4d71-bc07-a1f10e7c9f3f service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:38.171684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:38.172344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.172663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.172944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.184432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.185977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:38.199089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1228/2449] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:38.281129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:38.308841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:38.310442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:38.322343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41677d-8728-41d4-84d1-3db401c2092f service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:38.326326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:38.327604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.327898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.328182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.349731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d24eac-e3f1-4c65-aa04-c23f2dd9bc9c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:38.354448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:38.355465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.355752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.356035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] PATH_INFO: `/v3/users/110ca5f109264b64a79ad38aaa93e7f7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.357252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=110ca5f109264b64a79ad38aaa93e7f7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.359192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:38.391335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.393248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed200e9f-8d76-436a-8e9d-09c8ccb58865 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:38.393845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:38.397806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:38.404659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1229/2450] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:38.417490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110ca5f109264b64a79ad38aaa93e7f7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:38.425376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110ca5f109264b64a79ad38aaa93e7f7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:38.426046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110ca5f109264b64a79ad38aaa93e7f7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:38.437934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110ca5f109264b64a79ad38aaa93e7f7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:38.446571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110ca5f109264b64a79ad38aaa93e7f7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:38.453167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1222/2451] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:38 2022] DELETE /identity/v3/users/110ca5f109264b64a79ad38aaa93e7f7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:38.458013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.458310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.458635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.479728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:38.480982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:38.556577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1230/2452] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:38.566810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:38.568240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:38.627633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d81eb6-b1db-433d-ae49-3a19d350e1af admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:38.631194 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:38.631833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.632116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.632391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.641373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.642968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a81cb-e31c-4e86-857f-d97a460ab78d service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:38.652378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1223/2453] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:38.935116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:38.936564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:38.950791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe26419-c76f-443e-824c-150cd0edb9ab None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:38.956331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:38.957473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:38.957667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:38.957863 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] PATH_INFO: `/v3/projects/f522ae7a4fb74ca79ba67ced34d92c84` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:38.961011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f522ae7a4fb74ca79ba67ced34d92c84)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:38.961011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.020387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f522ae7a4fb74ca79ba67ced34d92c84', 'request_id': 'req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:39.026810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f522ae7a4fb74ca79ba67ced34d92c84', 'request_id': 'req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:39.034959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f522ae7a4fb74ca79ba67ced34d92c84', 'request_id': 'req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:39.046260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] The token cache is being invalidate because project f522ae7a4fb74ca79ba67ced34d92c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:39.046554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:39.047196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7dadbbaa-3cd7-4c1e-a31f-9fa07e715ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:39.049287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1231/2454] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:38 2022] DELETE /identity/v3/projects/f522ae7a4fb74ca79ba67ced34d92c84 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:39.054419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.054914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.055023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.073160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:39.074396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:39.133465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1224/2455] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:39.141763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.142063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.142405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.160879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:39.162085 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:39.170323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1232/2456] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:39.175391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.192709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfaf2b-6588-4574-ac59-0ccf38f66681 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.197418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.198160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.198448 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.198728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.199468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:39.199778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.201438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.215452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1225/2457] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:39.220035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.230979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b8396-3438-48e8-8b9c-ea4b8c6a65ef None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.235896 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.237715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.238030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.238295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.239026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.240969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.266996 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1233/2458] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:39.271606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.281619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88903b55-a4ad-42d7-8b3c-99398c6276ab admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.312793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:39.343623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.344301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.344762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.345044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.346259 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.348218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.369305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1226/2459] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:39.373813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.384980 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7280b25-b891-48da-af30-5685bea9036c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.390429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.391129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.391414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.391694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.392677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.394442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.406678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1234/2460] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:39.411635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.421770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e90b67-4627-470c-8708-c0d98579e058 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.427503 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.428171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.428489 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.428780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.429521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.431139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c149f4-e5bd-43a3-83cf-11f4ddf1d866 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.442198 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1227/2461] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:39.446623 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.457530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397f84f-5629-4579-8c66-a371373a7312 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.462279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.463166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.463447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.463721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] PATH_INFO: `/v3/projects/024767087ac848159055c4253391441e/users/6fbfd2cc286941f7a5e8eb727d16026a/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.480289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=024767087ac848159055c4253391441e, user_id=6fbfd2cc286941f7a5e8eb727d16026a, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.483005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.496038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1235/2462] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:39 2022] PUT /identity/v3/projects/024767087ac848159055c4253391441e/users/6fbfd2cc286941f7a5e8eb727d16026a/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:39.507897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.508187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.508486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.536242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] MFA Rules not processed for user `6fbfd2cc286941f7a5e8eb727d16026a`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:39.537476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:39.610026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1228/2463] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:39.614683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.635134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:39.684634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f027e-9fba-4a47-95b2-b6da08915d59 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.690258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.691087 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.691373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.691650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.692361 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.694332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.722976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1236/2464] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:39.727024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.738893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bf47be-6a92-4a81-9848-03ba401dd4d6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.800556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.801276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.801621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.801981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.802705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.804652 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.826697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1229/2465] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:39 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:39.830899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.844902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34690b-16df-4f56-b504-a7612a850a2f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.850446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.851200 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.851689 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.851931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.852699 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.854349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.872037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1237/2466] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:39.876425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.888757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc029c1d-c705-4110-a92f-23008de493f9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:39.893986 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:39.895097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:39.895438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:39.895849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] PATH_INFO: `/v3/projects/643b8be8b9de4bb08352d9343a2c33b3/users/d33d3d0c7b154f0db3af1887ce8cee7e/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:39.915476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643b8be8b9de4bb08352d9343a2c33b3, user_id=d33d3d0c7b154f0db3af1887ce8cee7e, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:39.918614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:39.936646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1230/2467] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:39 2022] PUT /identity/v3/projects/643b8be8b9de4bb08352d9343a2c33b3/users/d33d3d0c7b154f0db3af1887ce8cee7e/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:39.941802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:39.952061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e831b-85fa-4969-af10-da99623ab594 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:40.015272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:40.016152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.016646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.017057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.018262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:40.020082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d3500-2657-4441-9395-4d8239900a33 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:40.031717 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1238/2468] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:40.036684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:40.047063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f5471-cdb8-40a5-a538-87ffa05f2d93 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:40.051998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:40.052900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.053186 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.053465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] PATH_INFO: `/v3/projects/643b8be8b9de4bb08352d9343a2c33b3/users/d33d3d0c7b154f0db3af1887ce8cee7e/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.055590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643b8be8b9de4bb08352d9343a2c33b3, user_id=d33d3d0c7b154f0db3af1887ce8cee7e, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:40.058295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:40.060567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:40.071220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1231/2469] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:40 2022] PUT /identity/v3/projects/643b8be8b9de4bb08352d9343a2c33b3/users/d33d3d0c7b154f0db3af1887ce8cee7e/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:40.082011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.082302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.082583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.108552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] MFA Rules not processed for user `d33d3d0c7b154f0db3af1887ce8cee7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:40.109762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:40.179443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1239/2470] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:40.191668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:40.193534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:40.251355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f0144-c018-4e84-9071-31613c92a6b2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:40.255596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:40.256240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.256560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.256838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.266015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:40.267423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437478de-99eb-4008-8ad4-5577d8b9c14b service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:40.276793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1232/2471] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:40.414564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:40.429383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:40.484509 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb79aaa-83cf-40f0-a81f-54af149a81af None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:40.490035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:40.491011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.491370 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.492302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] PATH_INFO: `/v3/users/6fbfd2cc286941f7a5e8eb727d16026a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.493363 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbfd2cc286941f7a5e8eb727d16026a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:40.495449 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:40.555645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbfd2cc286941f7a5e8eb727d16026a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:40.563654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbfd2cc286941f7a5e8eb727d16026a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:40.565259 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbfd2cc286941f7a5e8eb727d16026a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:40.581666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbfd2cc286941f7a5e8eb727d16026a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:40.582493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbfd2cc286941f7a5e8eb727d16026a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:40.599429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1240/2472] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:40 2022] DELETE /identity/v3/users/6fbfd2cc286941f7a5e8eb727d16026a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:40.604092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.604616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.605081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.623943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:40.625314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:40.688117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1233/2473] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:40.698353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:40.699870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:40.758114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3791b88-6a45-40bf-8e44-5f7cc1750e94 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:40.762340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:40.762992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:40.763278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:40.763561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:40.773010 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:40.774446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:40.784983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1241/2474] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:41.088112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:41.089675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:41.112108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235603a-7c88-4ce1-9619-b356a98c0e7f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:41.119824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:41.120857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.121225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.121654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] PATH_INFO: `/v3/projects/024767087ac848159055c4253391441e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.123005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] RBAC: Authorizing `identity:delete_project(project_id=024767087ac848159055c4253391441e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:41.125124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:41.178697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024767087ac848159055c4253391441e', 'request_id': 'req-84fbbcf7-2642-4c60-b66e-061af242737a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.185522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024767087ac848159055c4253391441e', 'request_id': 'req-84fbbcf7-2642-4c60-b66e-061af242737a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.192955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024767087ac848159055c4253391441e', 'request_id': 'req-84fbbcf7-2642-4c60-b66e-061af242737a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.207255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] The token cache is being invalidate because project 024767087ac848159055c4253391441e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:41.207546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.208175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.210239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1234/2475] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:41 2022] DELETE /identity/v3/projects/024767087ac848159055c4253391441e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:41.214086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:41.225593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01937be8-3956-41bc-8cd2-fff75f80c0e6 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:41.261302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:41.289018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:41.289548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.289834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.290114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] PATH_INFO: `/v3/users/d33d3d0c7b154f0db3af1887ce8cee7e` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.291313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33d3d0c7b154f0db3af1887ce8cee7e)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:41.293668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:41.345393 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33d3d0c7b154f0db3af1887ce8cee7e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.353169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33d3d0c7b154f0db3af1887ce8cee7e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.354019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33d3d0c7b154f0db3af1887ce8cee7e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.367842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33d3d0c7b154f0db3af1887ce8cee7e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.368585 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-76a24374-2c6e-4126-97f4-a7eb108f62e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33d3d0c7b154f0db3af1887ce8cee7e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.383007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1242/2476] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:41 2022] DELETE /identity/v3/users/d33d3d0c7b154f0db3af1887ce8cee7e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:41.672820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:41.674605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:41.693120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbbcf7-2642-4c60-b66e-061af242737a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:41.712646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:41.757666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:41.758526 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.758807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.759082 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] PATH_INFO: `/v3/projects/643b8be8b9de4bb08352d9343a2c33b3` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.760239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=643b8be8b9de4bb08352d9343a2c33b3)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:41.762218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:41.808978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643b8be8b9de4bb08352d9343a2c33b3', 'request_id': 'req-222d88ba-f0d3-4348-a7d9-debf457baa8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.815811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643b8be8b9de4bb08352d9343a2c33b3', 'request_id': 'req-222d88ba-f0d3-4348-a7d9-debf457baa8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.821286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643b8be8b9de4bb08352d9343a2c33b3', 'request_id': 'req-222d88ba-f0d3-4348-a7d9-debf457baa8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.839351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] The token cache is being invalidate because project 643b8be8b9de4bb08352d9343a2c33b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:41.839634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.840275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-222d88ba-f0d3-4348-a7d9-debf457baa8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:41.841828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1235/2477] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:41 2022] DELETE /identity/v3/projects/643b8be8b9de4bb08352d9343a2c33b3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:41.847145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.847682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.847809 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.868494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:41.869622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:41.929507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1243/2478] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:41.937312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.937597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.937873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.956769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:41.957963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:41.965647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1236/2479] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:41.970067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:41.979935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e139f-0bf3-42e1-91f6-cbe59ac0c0a8 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:41.984569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:41.985218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:41.985501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:41.985773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:41.987088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:41.987395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:41.989101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.000537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1244/2480] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:42.005436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.014544 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e965e5-cecd-48e7-a325-a0be00e9777f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.019538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.020175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.020487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.020750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.021469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.023352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.049365 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1237/2481] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:42.053599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.063421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cf03a-51aa-4f15-87b0-83c5bd98566b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.103965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:42.122428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.123022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.123310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.123592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.124292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.126266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.146191 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1245/2482] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:42.150433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.161317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570113a9-284c-4c87-8a71-9e475dda8351 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.165877 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.166535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.166814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.167093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.167821 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.169732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.180935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1238/2483] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:42.184836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.194781 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca0e0-3e45-486c-b55d-2840598438c5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.199552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.200205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.200523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.200804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.201533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.203144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa570ba2-e252-4552-b448-7381fc7363ce admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.213677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1246/2484] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:42.217899 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.228746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdb29-54d7-40c8-ae59-a7bf088e6892 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.233426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.234294 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.234577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.234994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] PATH_INFO: `/v3/projects/df4e8fdb0117417bb70da2ddba46077f/users/51c7ecc5fbf941d19aa0f96d02e892e5/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.252010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4e8fdb0117417bb70da2ddba46077f, user_id=51c7ecc5fbf941d19aa0f96d02e892e5, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.254705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.266954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1239/2485] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:42 2022] PUT /identity/v3/projects/df4e8fdb0117417bb70da2ddba46077f/users/51c7ecc5fbf941d19aa0f96d02e892e5/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:42.278253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.278576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.278880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.305899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] MFA Rules not processed for user `51c7ecc5fbf941d19aa0f96d02e892e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:42.307106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:42.370580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1247/2486] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:42.375091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.425298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:42.441002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c3fb7-deb7-4021-8ed4-7bf2fbee1af0 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.445631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.446287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.446572 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.446849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.447549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.449496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.475466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1240/2487] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:42.479741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.490541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a0045-50ab-4a29-b16b-0fcce8265737 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.547868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.548624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.549064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.549373 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.550095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.552080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.574978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1248/2488] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:42.576236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.587154 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aca36a-444a-428d-909b-367c8040d5d7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.592275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.593236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.593524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.593802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.594536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.597436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.609947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1241/2489] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:42.613502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.624651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56217560-8bdb-4e60-98cd-f1bcffab36e1 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.630095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.630922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.631205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.631784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] PATH_INFO: `/v3/projects/b9360f45d09341718be907942db7c0da/users/36786386b9314f949bb9cfdfa6a95f5a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.651171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9360f45d09341718be907942db7c0da, user_id=36786386b9314f949bb9cfdfa6a95f5a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.654890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.671671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1249/2490] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:42 2022] PUT /identity/v3/projects/b9360f45d09341718be907942db7c0da/users/36786386b9314f949bb9cfdfa6a95f5a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:42.676209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.686867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e37ec-614b-4409-9919-4551bde10fab admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.752700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.753380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.753669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.753951 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.754682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.756309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728c91b-2e3a-4256-ab31-a3c56a978413 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.769575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1242/2491] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:42.770745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.781649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b8fbc-06df-4ddc-a64e-99de5b8ea3d4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:42.786338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:42.787225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.787525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.787898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] PATH_INFO: `/v3/projects/b9360f45d09341718be907942db7c0da/users/36786386b9314f949bb9cfdfa6a95f5a/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.789976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9360f45d09341718be907942db7c0da, user_id=36786386b9314f949bb9cfdfa6a95f5a, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:42.792685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:42.803988 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1250/2492] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:42 2022] PUT /identity/v3/projects/b9360f45d09341718be907942db7c0da/users/36786386b9314f949bb9cfdfa6a95f5a/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:42.815166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:42.815590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:42.815983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:42.851470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] MFA Rules not processed for user `36786386b9314f949bb9cfdfa6a95f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:42.852826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:42.927454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1243/2493] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:42.939816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:42.941562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:42.946846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:43.001028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0902c-d6f2-40a7-9414-396d7e24b964 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:43.004982 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:43.005625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:43.005913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:43.006188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:43.015195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:43.016700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:43.026665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1251/2494] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:43.337336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:43.347372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6415025-eb14-4e80-b73d-ec10719a84fc None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:43.352102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36786386b9314f949bb9cfdfa6a95f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9360f45d09341718be907942db7c0da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:43.352799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:43.353090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:43.353375 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:43.354102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:43.354420 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:43.356038 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:43.367727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1244/2495] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:43.371941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:43.381236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6254-d875-4b5c-aa9c-58a6b6c99217 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:43.386073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36786386b9314f949bb9cfdfa6a95f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9360f45d09341718be907942db7c0da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:43.386873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:43.387053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:43.387333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:43.388039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:43.390074 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:43.415911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1252/2496] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:43 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:43.420011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:43.430096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc5c9-fd70-47a3-ba8a-5b43dfda49c9 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:43.456662 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:43.488276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36786386b9314f949bb9cfdfa6a95f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9360f45d09341718be907942db7c0da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:43.488963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:43.489255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:43.489541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:43.490263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:43.490579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:43.492193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:43.503879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1245/2497] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:43.543061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:43.553631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62b48-6d4a-4998-82a3-dd33cd76fcea tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:43.558166 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36786386b9314f949bb9cfdfa6a95f5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9360f45d09341718be907942db7c0da', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:43.559014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:43.559301 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:43.559578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] PATH_INFO: `/v3/projects/61a8b253b64a404fbf55a496f4ecd1b2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:43.561522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorizing `identity:delete_project(project_id=61a8b253b64a404fbf55a496f4ecd1b2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:43.563652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:43.620841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a8b253b64a404fbf55a496f4ecd1b2', 'request_id': 'req-8eaa9980-bb80-42cf-95a7-8afafb894bb6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:43.628508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a8b253b64a404fbf55a496f4ecd1b2', 'request_id': 'req-8eaa9980-bb80-42cf-95a7-8afafb894bb6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:43.636250 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a8b253b64a404fbf55a496f4ecd1b2', 'request_id': 'req-8eaa9980-bb80-42cf-95a7-8afafb894bb6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:43.647691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] The token cache is being invalidate because project 61a8b253b64a404fbf55a496f4ecd1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:43.647978 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:43.648679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8eaa9980-bb80-42cf-95a7-8afafb894bb6 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:43.650230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1253/2498] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:43 2022] DELETE /identity/v3/projects/61a8b253b64a404fbf55a496f4ecd1b2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:43.917096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:43.918783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:44.001463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256c242-e615-44f0-a1a5-59b38d33ed99 tempest-VolumeQuotaClassesTest-1171241333 tempest-VolumeQuotaClassesTest-1171241333-project-admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:44.057940 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:44.058805 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:44.059084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:44.059356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] PATH_INFO: `/v3/users/51c7ecc5fbf941d19aa0f96d02e892e5` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:44.060577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c7ecc5fbf941d19aa0f96d02e892e5)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:44.062506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:44.108726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c7ecc5fbf941d19aa0f96d02e892e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.116356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c7ecc5fbf941d19aa0f96d02e892e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.117044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c7ecc5fbf941d19aa0f96d02e892e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.128969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:44.130141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c7ecc5fbf941d19aa0f96d02e892e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.138128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c7ecc5fbf941d19aa0f96d02e892e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.144542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1246/2499] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:43 2022] DELETE /identity/v3/users/51c7ecc5fbf941d19aa0f96d02e892e5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:44.149067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:44.149551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:44.149668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:44.166921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:44.168198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:44.229524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1254/2500] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:44.240975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:44.242550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:44.300608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90b4e-8111-4a19-9d8d-2157c89c1479 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:44.304199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:44.304887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:44.305174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:44.305453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:44.315297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:44.316756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:44.326123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1247/2501] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:44.591523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:44.592945 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:44.605401 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfbcd0-71d5-4779-9d86-3f94aea1eab4 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:44.609937 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:44.610820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:44.611208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:44.611486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] PATH_INFO: `/v3/projects/df4e8fdb0117417bb70da2ddba46077f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:44.612688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4e8fdb0117417bb70da2ddba46077f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:44.614620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:44.616311 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:44.663851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4e8fdb0117417bb70da2ddba46077f', 'request_id': 'req-c60614dd-7c3f-4a32-ac5b-760d83cd863a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.670015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4e8fdb0117417bb70da2ddba46077f', 'request_id': 'req-c60614dd-7c3f-4a32-ac5b-760d83cd863a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.677816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4e8fdb0117417bb70da2ddba46077f', 'request_id': 'req-c60614dd-7c3f-4a32-ac5b-760d83cd863a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.688843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] The token cache is being invalidate because project df4e8fdb0117417bb70da2ddba46077f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:44.689704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.690646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.692346 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1255/2502] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:44 2022] DELETE /identity/v3/projects/df4e8fdb0117417bb70da2ddba46077f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:44.696263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:44.706916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6ef22-aaac-4e44-a238-6d7e5c169a80 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:44.763982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:44.765073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:44.765314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:44.765585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] PATH_INFO: `/v3/users/36786386b9314f949bb9cfdfa6a95f5a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:44.766921 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=36786386b9314f949bb9cfdfa6a95f5a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:44.769137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:44.815939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36786386b9314f949bb9cfdfa6a95f5a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.823034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36786386b9314f949bb9cfdfa6a95f5a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.823751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36786386b9314f949bb9cfdfa6a95f5a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.835778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36786386b9314f949bb9cfdfa6a95f5a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.843875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c88bf14d-c038-4385-8c50-9ca4cfaffa4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36786386b9314f949bb9cfdfa6a95f5a'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:44.850031 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1248/2503] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:44 2022] DELETE /identity/v3/users/36786386b9314f949bb9cfdfa6a95f5a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:45.148880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.151564 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:45.167853 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60614dd-7c3f-4a32-ac5b-760d83cd863a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.235859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.237540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.237908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.238170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] PATH_INFO: `/v3/projects/b9360f45d09341718be907942db7c0da` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.239730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9360f45d09341718be907942db7c0da)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.242246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.291410 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9360f45d09341718be907942db7c0da', 'request_id': 'req-c759dcb7-f0c8-4411-bc1d-786f8f23e508'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:45.297507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9360f45d09341718be907942db7c0da', 'request_id': 'req-c759dcb7-f0c8-4411-bc1d-786f8f23e508'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:45.305931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9360f45d09341718be907942db7c0da', 'request_id': 'req-c759dcb7-f0c8-4411-bc1d-786f8f23e508'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:45.317359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] The token cache is being invalidate because project b9360f45d09341718be907942db7c0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:45.317654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:45.318356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-c759dcb7-f0c8-4411-bc1d-786f8f23e508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:45.320170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1256/2504] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:45 2022] DELETE /identity/v3/projects/b9360f45d09341718be907942db7c0da => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:45.325522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.325808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.326081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.344293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:45.346234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:45.352331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:45.412905 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1249/2505] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:45.422471 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.422907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.423195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.442553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:45.443842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:45.452251 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1257/2506] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:45.457091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.466937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9bc37-c475-4ada-a62c-0e4061a3432c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.472493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.473159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.473449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.473727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.474508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:45.474820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.476533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.489126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1250/2507] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:45.493390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.503541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a1fa9-86ad-4061-849c-75d3e76bdd55 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.508240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.508920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.509205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.509504 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.510202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.512121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.522788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:45.539468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1258/2508] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:45.543883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.554717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b3381-7205-4946-9145-09d4f4f42978 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.612236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.612928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.613220 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.613507 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.614209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.616160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.636856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1251/2509] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:45.640970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.651379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b00910-6f04-4e61-b3dc-94095e04d223 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.655957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.656650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.656938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.657222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.658162 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.659811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.670117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1259/2510] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:45.675101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.685007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ea640-c2cb-49a8-b317-8b3df41540ba admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.690862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.691518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.691801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.692076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.692883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.694428 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5fec1c-ea79-4c57-8c75-8e4f970fb90c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.704666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1252/2511] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:45.709076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.719240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdc9be-7281-448f-823b-7c7236fa854e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.723920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.724842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.725146 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.725407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] PATH_INFO: `/v3/projects/26a4825c97bb48308d61be3c37b1a152/users/9d7e607651f440af89a47b0d465109c0/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.740339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a4825c97bb48308d61be3c37b1a152, user_id=9d7e607651f440af89a47b0d465109c0, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.743064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.756818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1260/2512] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:45 2022] PUT /identity/v3/projects/26a4825c97bb48308d61be3c37b1a152/users/9d7e607651f440af89a47b0d465109c0/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:45.767630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.768015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.768604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.795328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] MFA Rules not processed for user `9d7e607651f440af89a47b0d465109c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:45.796555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:45.865478 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1253/2513] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:45.869958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.932245 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e392d5-2af0-47af-b6e3-8d054b32cb38 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.936948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:45.937604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:45.937893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:45.938169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:45.938855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:45.941428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:45.966491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1261/2514] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:45.971334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:45.982175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0abcef-016b-4fed-a8f9-425980fc2b24 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:45.985736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:46.039400 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.040065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.040351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.040659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.041342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.043266 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.063950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1254/2515] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:45 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:46.067898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.078865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930773d-8e4e-4dd1-b76a-a8b7551c1b5e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.083872 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.084575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.084865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.085150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.085883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.087501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.099350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1262/2516] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:46.103628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.114236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc63fbe-cea3-4933-99ad-d0661e013281 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.119079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.119966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.120251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.120596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] PATH_INFO: `/v3/projects/78ba3692ea594760967bade63a79d8d4/users/d0c746469ed744549ffc1f71a87e8015/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.135795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ba3692ea594760967bade63a79d8d4, user_id=d0c746469ed744549ffc1f71a87e8015, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.138512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.152148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1255/2517] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:46 2022] PUT /identity/v3/projects/78ba3692ea594760967bade63a79d8d4/users/d0c746469ed744549ffc1f71a87e8015/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:46.156099 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.167356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869e10f-7e41-4b6f-84be-025cec9cecbe admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.231892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.232600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.232885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.233164 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.233983 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.235826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373127a-f706-45f3-93b6-2899cf9c3a95 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.249084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1263/2518] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:46.253382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.263717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109245d4-3fd0-4ae8-b31e-fb8664884c46 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.269306 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.270174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.270588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.270870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] PATH_INFO: `/v3/projects/78ba3692ea594760967bade63a79d8d4/users/d0c746469ed744549ffc1f71a87e8015/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.273188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ba3692ea594760967bade63a79d8d4, user_id=d0c746469ed744549ffc1f71a87e8015, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.276114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.288829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1256/2519] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:46 2022] PUT /identity/v3/projects/78ba3692ea594760967bade63a79d8d4/users/d0c746469ed744549ffc1f71a87e8015/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:46.299460 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.299778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.300160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.337582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] MFA Rules not processed for user `d0c746469ed744549ffc1f71a87e8015`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:46.338852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:46.353582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:46.431736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1264/2520] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:46.443450 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.445482 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:46.521550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75ede7-f689-4d91-b74e-126c97947ef9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.526510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.527314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.527708 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.527884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.545673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.547086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.558811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1257/2521] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:46.656883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:46.658296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:46.670000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee19a6f-17e9-4333-9d1e-f9b14fd1e858 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:46.673637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:46.674278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:46.674566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:46.674848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:46.735284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:46.736891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39654f96-ec0c-451b-88bb-732331e9831c service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:46.745887 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1265/2522] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:47.791171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:47.792650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:47.859588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1664bb-fab6-4419-9289-197e6438801b service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:47.864151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:47.865054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:47.865335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:47.865608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] PATH_INFO: `/v3/users/7e2ca3a5f8ea4cf48421ae85db2999a5` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:47.866783 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2ca3a5f8ea4cf48421ae85db2999a5)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:47.868740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:47.870970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:47.916836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2ca3a5f8ea4cf48421ae85db2999a5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:47.923540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2ca3a5f8ea4cf48421ae85db2999a5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:47.924247 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2ca3a5f8ea4cf48421ae85db2999a5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:47.935800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2ca3a5f8ea4cf48421ae85db2999a5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:47.942404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2ca3a5f8ea4cf48421ae85db2999a5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:47.948310 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1258/2523] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:47 2022] DELETE /identity/v3/users/7e2ca3a5f8ea4cf48421ae85db2999a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:47.952907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:47.953369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:47.953464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:47.971568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:47.972820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:48.031169 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1266/2524] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:48.041781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:48.043119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:48.110184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdb791-a1ac-4ebd-b3e8-15086577ad31 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:48.114163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:48.114812 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:48.115097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:48.115372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:48.124660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:48.126138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:48.136527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1259/2525] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:48.437984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:48.440039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:48.453131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee2df9-d8b5-4f74-9849-6bc28c5a10ba None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:48.458668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:48.459710 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:48.460116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:48.460500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] PATH_INFO: `/v3/projects/5c31441c29c54f7789ccd461adc47d2f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:48.462016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c31441c29c54f7789ccd461adc47d2f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:48.464487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:48.509574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:48.512720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c31441c29c54f7789ccd461adc47d2f', 'request_id': 'req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.520161 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c31441c29c54f7789ccd461adc47d2f', 'request_id': 'req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.527955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c31441c29c54f7789ccd461adc47d2f', 'request_id': 'req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.539732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] The token cache is being invalidate because project 5c31441c29c54f7789ccd461adc47d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:48.540017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.540669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.542240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1267/2526] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:48 2022] DELETE /identity/v3/projects/5c31441c29c54f7789ccd461adc47d2f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:48.546245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:48.556866 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ceff3-d853-41c1-913a-018e338ac85d service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:48.614235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:48.615136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:48.615419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:48.615694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] PATH_INFO: `/v3/users/8a0c4cbed69f4f67bc2573f51983ab3c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:48.616931 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0c4cbed69f4f67bc2573f51983ab3c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:48.618870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:48.646891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:48.666510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0c4cbed69f4f67bc2573f51983ab3c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.673824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0c4cbed69f4f67bc2573f51983ab3c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.674494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0c4cbed69f4f67bc2573f51983ab3c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.686585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0c4cbed69f4f67bc2573f51983ab3c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.693839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-13d2219f-a4e5-48b4-8d7c-61200c612f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0c4cbed69f4f67bc2573f51983ab3c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:48.699875 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1260/2527] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:48 2022] DELETE /identity/v3/users/8a0c4cbed69f4f67bc2573f51983ab3c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:49.052273 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.053801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:49.069082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5f5fc-2d44-45ae-b673-d4cefa1cb59b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.135953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.136852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.137167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.137559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] PATH_INFO: `/v3/projects/d449019765dc48dc90ed127d5171b91a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.138757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d449019765dc48dc90ed127d5171b91a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.140746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.187298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd449019765dc48dc90ed127d5171b91a', 'request_id': 'req-0518c011-e86d-4caa-b341-e9f2acb7a7bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:49.193700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd449019765dc48dc90ed127d5171b91a', 'request_id': 'req-0518c011-e86d-4caa-b341-e9f2acb7a7bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:49.206741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd449019765dc48dc90ed127d5171b91a', 'request_id': 'req-0518c011-e86d-4caa-b341-e9f2acb7a7bd'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:49.218825 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] The token cache is being invalidate because project d449019765dc48dc90ed127d5171b91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:49.219122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:49.219764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0518c011-e86d-4caa-b341-e9f2acb7a7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:49.222048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1268/2528] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:49 2022] DELETE /identity/v3/projects/d449019765dc48dc90ed127d5171b91a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:49.227104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.227559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.227700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.247511 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:49.248900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:49.309460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1261/2529] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:49.317482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.317776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.318061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.344583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:49.346645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:49.354939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1269/2530] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:49.359515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.369292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3a52-7bc2-4703-b4bc-1c60d646f8a2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.374078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.374800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.375093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.375366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.376104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:49.376496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.378074 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.398468 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1262/2531] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:49.400936 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.411608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be22e0-98e9-41a1-9966-2e59201ee184 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.416847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.417502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.417790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.418575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.419295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.421243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.448290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1270/2532] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:49.452307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.462971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ac3-84f1-44f5-affc-e6c7c5dacb9b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.526134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.526814 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.527140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.527560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.528309 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.530277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.555130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1263/2533] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:49.556354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.566835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7b0a-3077-48f7-937f-2b9dc8b61001 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.571647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.572312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.572630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.572913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.573639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.575467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.589313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1271/2534] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:49.593573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.605943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a5e07-4f93-40fe-ad43-e6f68a91394d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.610672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.611631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.611916 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.612263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.612954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.614555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554ab25-270a-46f4-a613-78c3ac840a97 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.625329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1264/2535] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:49.631036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.641473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d239-9e29-46a3-817d-b9ab9dd515ec admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.647490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.648502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.648865 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.649145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] PATH_INFO: `/v3/projects/68e6b818d2e146f0b50dd57057ee3628/users/789fc90a3ba347ccb00e0a1ef1f034de/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.665019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e6b818d2e146f0b50dd57057ee3628, user_id=789fc90a3ba347ccb00e0a1ef1f034de, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.667831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.678190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:49.687056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1272/2536] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:49 2022] PUT /identity/v3/projects/68e6b818d2e146f0b50dd57057ee3628/users/789fc90a3ba347ccb00e0a1ef1f034de/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:49.702452 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.702849 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.703246 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.724236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:49.736322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] MFA Rules not processed for user `789fc90a3ba347ccb00e0a1ef1f034de`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:49.738057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:49.816079 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1265/2537] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:49.821226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.887771 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d77c88-647a-404b-805d-f9e4116a3fa2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:49.897382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:49.898055 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:49.898342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:49.898624 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:49.899327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:49.901377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:49.932359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1273/2538] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:49.933542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:49.944835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb580e10-5be3-41ca-99e1-34ac4ace975c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.009069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.009750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.010039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.010318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.011009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.012991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.038818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1266/2539] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:49 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:50.040075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:50.053890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692f760-4e1c-41a0-b9a1-4811062eef25 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.056923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.057788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.058078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.058417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.059152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.060881 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.073590 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1274/2540] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:50.078048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:50.088575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca32cb-5ff5-4085-95f4-b895b01b5c62 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.093591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.094479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.094793 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.095069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] PATH_INFO: `/v3/projects/93f333178954445da66832abc388ca75/users/2ba75b062ca043c5be39b3e41c5dbab5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.111277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f333178954445da66832abc388ca75, user_id=2ba75b062ca043c5be39b3e41c5dbab5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.114308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.130006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1267/2541] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:50 2022] PUT /identity/v3/projects/93f333178954445da66832abc388ca75/users/2ba75b062ca043c5be39b3e41c5dbab5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:50.134176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:50.150466 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b0f93-99c5-48ab-b1d6-2e2f6bdd7e79 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.216876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.217548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.217910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.218355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.219083 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.220755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a197e-f58f-41c5-9278-6bed81e25cc6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.232133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1275/2542] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:50.239034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:50.250289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85742550-5680-44c0-9cba-b27603b47fb3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.258441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.259344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.259606 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.259891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] PATH_INFO: `/v3/projects/93f333178954445da66832abc388ca75/users/2ba75b062ca043c5be39b3e41c5dbab5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.261893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f333178954445da66832abc388ca75, user_id=2ba75b062ca043c5be39b3e41c5dbab5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.264585 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.275961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1268/2543] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:50 2022] PUT /identity/v3/projects/93f333178954445da66832abc388ca75/users/2ba75b062ca043c5be39b3e41c5dbab5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:50.287633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.287927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.288217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.317615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] MFA Rules not processed for user `2ba75b062ca043c5be39b3e41c5dbab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:50.318972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:50.384815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:50.395812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1276/2544] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:50.406962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:50.409056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:50.468475 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eff7f-6933-4b3f-a3d1-ba56f03dfe4e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:50.473513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:50.474376 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:50.474701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:50.474985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:50.481226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:50.485768 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:50.488081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bc5f0-04ab-4750-a249-d7deb227c9e6 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:50.499210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1269/2545] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:51.547985 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:51.549759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:51.622909 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f79289-9e53-4a86-83e4-57e5c8942b3f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:51.628814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:51.629874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:51.630160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:51.630441 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] PATH_INFO: `/v3/users/9d7e607651f440af89a47b0d465109c0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:51.631752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7e607651f440af89a47b0d465109c0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:51.633621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:51.682038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7e607651f440af89a47b0d465109c0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.689719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7e607651f440af89a47b0d465109c0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.690648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7e607651f440af89a47b0d465109c0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.703594 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7e607651f440af89a47b0d465109c0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.704277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7e607651f440af89a47b0d465109c0'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.718883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1277/2546] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:51 2022] DELETE /identity/v3/users/9d7e607651f440af89a47b0d465109c0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:51.723429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:51.723927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:51.724026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:51.742936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:51.744170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:51.813285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1270/2547] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:51.823268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:51.824790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:51.833854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:51.835341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:51.848754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b883a1-5cdf-47e9-bd53-44c4dfd5d015 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:51.853412 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:51.854277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:51.854560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:51.854838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] PATH_INFO: `/v3/users/789fc90a3ba347ccb00e0a1ef1f034de` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:51.856040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789fc90a3ba347ccb00e0a1ef1f034de)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:51.858010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:51.886792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7546d529-874c-4186-98af-afec6961c9cd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:51.891309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:51.892511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:51.892906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:51.893289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:51.903558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:51.905097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b576d-a8b6-42d2-bc10-b5f54db5ebe0 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:51.906356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789fc90a3ba347ccb00e0a1ef1f034de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.914125 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789fc90a3ba347ccb00e0a1ef1f034de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.914813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789fc90a3ba347ccb00e0a1ef1f034de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.915557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1278/2548] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:51.928058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789fc90a3ba347ccb00e0a1ef1f034de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.935871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789fc90a3ba347ccb00e0a1ef1f034de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:51.942471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1271/2549] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:51 2022] DELETE /identity/v3/users/789fc90a3ba347ccb00e0a1ef1f034de => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:51.947313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:51.947629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:51.947923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:51.965811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:51.967150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:51.980314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:52.031547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1279/2550] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:52.041806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.043202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:52.104114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3dcbc-20da-4760-a56b-87e7b44617a8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:52.107780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:52.108430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:52.108740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:52.109014 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:52.118806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:52.120244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:52.131087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1272/2551] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:52.236627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.238484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:52.252212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11219741-c6b2-4a9f-9d8e-8e356a41bc89 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:52.268815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:52.270061 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:52.270344 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:52.270697 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] PATH_INFO: `/v3/projects/26a4825c97bb48308d61be3c37b1a152` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:52.272049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a4825c97bb48308d61be3c37b1a152)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:52.274423 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:52.327647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a4825c97bb48308d61be3c37b1a152', 'request_id': 'req-d9a973f2-c4f5-4064-b595-59185b5e3ab5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.335326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a4825c97bb48308d61be3c37b1a152', 'request_id': 'req-d9a973f2-c4f5-4064-b595-59185b5e3ab5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.344253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a4825c97bb48308d61be3c37b1a152', 'request_id': 'req-d9a973f2-c4f5-4064-b595-59185b5e3ab5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.355784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] The token cache is being invalidate because project 26a4825c97bb48308d61be3c37b1a152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:52.356211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.356933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.358413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1280/2552] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:52 2022] DELETE /identity/v3/projects/26a4825c97bb48308d61be3c37b1a152 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:52.363109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.373182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6201911-ca80-48f9-8905-10d589f268aa service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:52.381030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:52.441380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:52.443305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:52.443305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:52.443305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] PATH_INFO: `/v3/users/d0c746469ed744549ffc1f71a87e8015` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:52.444408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c746469ed744549ffc1f71a87e8015)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:52.446546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:52.482306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.487765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:52.508898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a973f2-c4f5-4064-b595-59185b5e3ab5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:52.515265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:52.516693 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:52.517208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:52.518435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] PATH_INFO: `/v3/projects/68e6b818d2e146f0b50dd57057ee3628` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:52.518720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e6b818d2e146f0b50dd57057ee3628)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:52.520767 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:52.529480 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c746469ed744549ffc1f71a87e8015'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.537573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c746469ed744549ffc1f71a87e8015'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.537906 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c746469ed744549ffc1f71a87e8015'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.552192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c746469ed744549ffc1f71a87e8015'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.561415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c746469ed744549ffc1f71a87e8015'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.570559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1273/2553] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:52 2022] DELETE /identity/v3/users/d0c746469ed744549ffc1f71a87e8015 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:52.577188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e6b818d2e146f0b50dd57057ee3628', 'request_id': 'req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.584768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e6b818d2e146f0b50dd57057ee3628', 'request_id': 'req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.593565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e6b818d2e146f0b50dd57057ee3628', 'request_id': 'req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.606268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] The token cache is being invalidate because project 68e6b818d2e146f0b50dd57057ee3628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:52.606566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.607640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.612779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1281/2554] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:52 2022] DELETE /identity/v3/projects/68e6b818d2e146f0b50dd57057ee3628 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:52.613989 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.615455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:52.692144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5689bba-1399-4348-b900-b81a266e0876 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:52.696946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:52.697937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:52.698281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:52.698564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] PATH_INFO: `/v3/users/2ba75b062ca043c5be39b3e41c5dbab5` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:52.700221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba75b062ca043c5be39b3e41c5dbab5)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:52.702215 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:52.750760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:52.765318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba75b062ca043c5be39b3e41c5dbab5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.772638 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba75b062ca043c5be39b3e41c5dbab5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.773301 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba75b062ca043c5be39b3e41c5dbab5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.786747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba75b062ca043c5be39b3e41c5dbab5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.794161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba75b062ca043c5be39b3e41c5dbab5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:52.801142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1274/2555] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:52 2022] DELETE /identity/v3/users/2ba75b062ca043c5be39b3e41c5dbab5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:52.870229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:52.872290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:52.904573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:52.949243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea323e-cb65-44d8-a84b-a7ddcfdec5b8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.011972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.012899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.013185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.013462 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] PATH_INFO: `/v3/projects/78ba3692ea594760967bade63a79d8d4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.014651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ba3692ea594760967bade63a79d8d4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.016620 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.063875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ba3692ea594760967bade63a79d8d4', 'request_id': 'req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.070856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ba3692ea594760967bade63a79d8d4', 'request_id': 'req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.080934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ba3692ea594760967bade63a79d8d4', 'request_id': 'req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.087481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.089532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.092902 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] The token cache is being invalidate because project 78ba3692ea594760967bade63a79d8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:53.093457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.097173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-0bf37bf1-e5d7-4445-8663-02c6e6621b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.099045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1282/2556] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:52 2022] DELETE /identity/v3/projects/78ba3692ea594760967bade63a79d8d4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:53.104653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d5cc-4a00-48ea-887e-df200680cc73 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.104653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d5cc-4a00-48ea-887e-df200680cc73 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.104972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d5cc-4a00-48ea-887e-df200680cc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.109536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122f42-b517-4c85-af95-d0c294c81662 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.125885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5d50d5cc-4a00-48ea-887e-df200680cc73 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.127779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5d50d5cc-4a00-48ea-887e-df200680cc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.172366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.174085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.174367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.174642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] PATH_INFO: `/v3/projects/93f333178954445da66832abc388ca75` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.175859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93f333178954445da66832abc388ca75)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.177863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.187330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1283/2557] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:53.197420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0446-860c-474a-b5d3-e2449977785b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.197629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0446-860c-474a-b5d3-e2449977785b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.197890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0446-860c-474a-b5d3-e2449977785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.204201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:53.221812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-788c0446-860c-474a-b5d3-e2449977785b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.223013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-788c0446-860c-474a-b5d3-e2449977785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.229084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f333178954445da66832abc388ca75', 'request_id': 'req-414081ef-0c0e-49a2-ac7d-83f6ea762819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.237373 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f333178954445da66832abc388ca75', 'request_id': 'req-414081ef-0c0e-49a2-ac7d-83f6ea762819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.244610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f333178954445da66832abc388ca75', 'request_id': 'req-414081ef-0c0e-49a2-ac7d-83f6ea762819'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.258327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] The token cache is being invalidate because project 93f333178954445da66832abc388ca75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:53.258617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.259293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-414081ef-0c0e-49a2-ac7d-83f6ea762819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:53.260887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1275/2558] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:53 2022] DELETE /identity/v3/projects/93f333178954445da66832abc388ca75 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:53.265633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0d6-5997-4bab-91b1-e5ce5a0f6349 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.265982 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0d6-5997-4bab-91b1-e5ce5a0f6349 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.266323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99b0d6-5997-4bab-91b1-e5ce5a0f6349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.284338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1284/2559] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:53.286364 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-9d99b0d6-5997-4bab-91b1-e5ce5a0f6349 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.287596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-9d99b0d6-5997-4bab-91b1-e5ce5a0f6349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.289994 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0446-860c-474a-b5d3-e2449977785b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.295514 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1276/2560] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:53.304726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.305010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.305287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.305702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0446-860c-474a-b5d3-e2449977785b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.310644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.311419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.311709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.312212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.313032 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:53.313330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.315567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.324058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.325443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.327653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1285/2561] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:18:53.332253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.333702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1277/2562] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:53.337839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.342242 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dfc01-e877-4a89-b2cf-276905edd7ed admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.347647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb245d40-c06a-4890-b22e-4b5585450e26 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.347778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.348335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.348650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.349060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.349745 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.351647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.352639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.353289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.353576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.353856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.354581 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:53.354895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.356709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.370356 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1278/2563] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:18:53.374816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.383744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1286/2564] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:53.386655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731db0d-c2c7-45a5-9c77-2a8dc2f98561 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.388020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.395683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:53.449941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.450711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.451002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.451283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.451983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.452631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e900cdc-587b-423a-8e3c-3f80b1ba45e8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.453934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.457404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.458066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.458348 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.458651 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.459360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.461299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.483536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1279/2565] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:53.487802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1287/2566] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:53.487942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.488759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.498332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679354a-a32c-4604-824e-46d9063c9bae admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.548280 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7a827-9ad5-45fc-b6ab-c7bd7c52f8fd admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.551518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:53.553647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.554497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.554781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.555069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.555873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.558010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.558451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.559299 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.559752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.560216 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.560992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.562580 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.573715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1288/2567] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:53.577622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.579064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1280/2568] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:53.583212 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.588193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafd49-02fe-46a9-a29b-f6f6634198ca admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.593583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917d01a-d802-4d2c-82cd-e954cd3cdac9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.596658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.597442 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.597688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.598044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.599007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.599789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.600477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.600599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.600796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.601117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.602075 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.603679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.615425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1289/2569] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:53.615628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1281/2570] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:53.616827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.618740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.626943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d28ebc-9b2a-4c2f-944e-ecac1faa87cd admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.628432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37e188-e518-4e16-89e6-3af030ecee7f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.632644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.633615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.633775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.633890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.634158 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] PATH_INFO: `/v3/projects/770da3da46e742de95a26d1ce0be6f0a/users/cbb63f6d0d104c21958bd6435bfa8cf2/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.634419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.634704 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.634984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.635827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.637449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.651558 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1282/2571] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:53.651700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=770da3da46e742de95a26d1ce0be6f0a, user_id=cbb63f6d0d104c21958bd6435bfa8cf2, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.653559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.653923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9824b2-acb2-4a4e-923c-4e09edc7cbcf admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.667891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8351e1-858c-43fc-b1fa-5791cc7ff694 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.669060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1290/2572] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:53 2022] PUT /identity/v3/projects/770da3da46e742de95a26d1ce0be6f0a/users/cbb63f6d0d104c21958bd6435bfa8cf2/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:53.679723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.680016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.680284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.710503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] MFA Rules not processed for user `cbb63f6d0d104c21958bd6435bfa8cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.711728 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.741260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.742124 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.742407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.742680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] PATH_INFO: `/v3/projects/9227eb3062d9431c91fd5a1bb7dc93d7/users/b2372943b0944bd3a8092fbc59e72561/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.750996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:53.759985 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9227eb3062d9431c91fd5a1bb7dc93d7, user_id=b2372943b0944bd3a8092fbc59e72561, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.762754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2622bc-3ce2-4827-8612-06b4327a2ee8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.779282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1283/2573] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:53 2022] PUT /identity/v3/projects/9227eb3062d9431c91fd5a1bb7dc93d7/users/b2372943b0944bd3a8092fbc59e72561/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:53.788939 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1291/2574] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:53.792402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.792920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.793139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.793248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.804611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5331642-6243-4279-9b4c-cb24b5ef4609 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.825074 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] MFA Rules not processed for user `b2372943b0944bd3a8092fbc59e72561`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:53.826382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:53.866168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.866840 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.867126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.867405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.868104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.870115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.893727 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1284/2575] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:53.898390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.899310 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1292/2576] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:53.903076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:53.909604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6c3b1-6660-4731-96cf-f671b918afe3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.936830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:53.965575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.966286 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.966571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.966920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.967618 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.967949 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b539df-02ff-4991-ad9c-fc5e788915ea admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:53.969829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.973637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:53.974684 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:53.974977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:53.975260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:53.976093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:53.978067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:53.999745 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1285/2577] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:53.999953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1293/2578] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:53 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:54.001435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.002546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.012193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862bcd-0fa9-4282-81a1-50f4bf97e5d8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.065029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4db7ba-d22a-458b-9426-dfa17f4944fb admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.069828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.070071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.070491 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.070736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.070835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.071025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.071134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.071313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.071750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.071999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.073388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.073954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.081957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:54.083305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1294/2579] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:54.087355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.101112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1286/2580] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:54.101291 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf12ea8-7163-426e-9777-68b90d6c45a8 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.102267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.103117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.103772 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.104058 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.104339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.105089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.106704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.112590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c1a52-0bf5-43fc-a3d4-91f4c43d9a71 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.120047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1295/2581] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:54.120209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.120209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.120209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.120209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.120623 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.120904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.121050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.131828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6186ec-d245-4b02-8094-5ecb210c9370 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.132517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1287/2582] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:54.136856 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.137391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.138342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.138631 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.138918 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] PATH_INFO: `/v3/projects/e65a963e87a243088d186a3446dd2e6c/users/aa44368cde984ec280692ac34a57a7af/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.147775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3a83-ee92-4479-b56b-1777c6b71222 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.153037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.153922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.154205 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.154483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] PATH_INFO: `/v3/projects/d34f840e1f684231952671021a34a288/users/42b8f108064b4124b81ea743ec0ce084/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.157090 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65a963e87a243088d186a3446dd2e6c, user_id=aa44368cde984ec280692ac34a57a7af, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.159816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921ddbb-cee7-44e9-b0f5-14b9e9baa59a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.169632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34f840e1f684231952671021a34a288, user_id=42b8f108064b4124b81ea743ec0ce084, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.172327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.174247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1296/2583] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:54 2022] PUT /identity/v3/projects/e65a963e87a243088d186a3446dd2e6c/users/aa44368cde984ec280692ac34a57a7af/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:54.185168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.185473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.185755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.189308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1288/2584] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:54 2022] PUT /identity/v3/projects/d34f840e1f684231952671021a34a288/users/42b8f108064b4124b81ea743ec0ce084/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:54.193268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.218005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] MFA Rules not processed for user `aa44368cde984ec280692ac34a57a7af`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:54.219482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.266440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8187a-bed5-4cdf-abdd-801ed9f09735 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.272032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.272728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.273021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.273302 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.274029 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.275687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.285731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1297/2585] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:54.289224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1289/2586] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:54.290192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.290309 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.299939 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f75a-8768-41ec-a4a5-c3057acbf740 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.301023 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b9bc8-7d5f-4dfd-b987-ab88af55e2f2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.304423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.305314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.305540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.305667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.305863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] PATH_INFO: `/v3/projects/d34f840e1f684231952671021a34a288/users/42b8f108064b4124b81ea743ec0ce084/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.306180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.306463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.306742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.307427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.307859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34f840e1f684231952671021a34a288, user_id=42b8f108064b4124b81ea743ec0ce084, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.309649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.310625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6104f-a7bd-4f2c-851d-5084b95fa822 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.320377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:54.322589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1290/2587] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:54 2022] PUT /identity/v3/projects/d34f840e1f684231952671021a34a288/users/42b8f108064b4124b81ea743ec0ce084/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:54.334193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.334479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.334759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.339730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1298/2588] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:54.343847 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.362372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] MFA Rules not processed for user `42b8f108064b4124b81ea743ec0ce084`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:54.363620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.411382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2be53c-e5b2-47cb-8908-5e49070b061b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.416408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.417184 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.417383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.417661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.418351 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.420975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.436175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1291/2589] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:54.443681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1299/2590] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:54.447278 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.448425 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.450082 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.457904 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060721d-b3c1-4871-9742-cf53c92af931 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.462537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.463192 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.463563 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.463748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.464497 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.466105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.476506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1292/2591] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:54.480576 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.491682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdf896-aca7-4a91-9cd8-60c14ba6c138 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.496791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.497666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.497941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.498339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] PATH_INFO: `/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac/users/f514db5c26ff40088acba62205df9ce2/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.509700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:54.510465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d9324-28f2-4d85-a38c-2eb7c5ec2586 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.514469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd2f3ec019a43adbe1560a2bd0031ac, user_id=f514db5c26ff40088acba62205df9ce2, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.514617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.515380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.515660 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.515941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.519096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.526235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.527875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.533529 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1293/2592] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:54 2022] PUT /identity/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac/users/f514db5c26ff40088acba62205df9ce2/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:54.537303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.537851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1300/2593] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:54.604930 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.607102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4926ac-fda5-4866-b14d-76796edd5e94 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.611952 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.612657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.612946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.613223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.613950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.615527 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d05b68-3df3-4215-8169-b3918ae0230e service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.615646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.620224 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.621121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.621408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.621690 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] PATH_INFO: `/v3/users/b2372943b0944bd3a8092fbc59e72561` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.622946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2372943b0944bd3a8092fbc59e72561)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.624926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.625713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1294/2594] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:54.630317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.640346 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03103864-d8b2-4cd2-91c0-0f705953f5c0 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.645981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.646864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.647148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.647443 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] PATH_INFO: `/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac/users/f514db5c26ff40088acba62205df9ce2/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.649431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd2f3ec019a43adbe1560a2bd0031ac, user_id=f514db5c26ff40088acba62205df9ce2, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.652085 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461321f4-e176-468f-a307-3e8abdcb72e9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.662874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1295/2595] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:54 2022] PUT /identity/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac/users/f514db5c26ff40088acba62205df9ce2/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:54.673707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.674018 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.674303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.675104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2372943b0944bd3a8092fbc59e72561'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:54.682873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2372943b0944bd3a8092fbc59e72561'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:54.683744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2372943b0944bd3a8092fbc59e72561'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:54.697020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2372943b0944bd3a8092fbc59e72561'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:54.697694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2e24c429-b06a-4afb-9317-95ef0660f3cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2372943b0944bd3a8092fbc59e72561'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:54.701098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] MFA Rules not processed for user `f514db5c26ff40088acba62205df9ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:54.702312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.702906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:54.713329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1301/2596] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:54 2022] DELETE /identity/v3/users/b2372943b0944bd3a8092fbc59e72561 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:54.717483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.717777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.718308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.739264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:54.740531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.773695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1296/2597] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:54.784711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.786231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.800766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1302/2598] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:54.810554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:54.811950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.846970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9358bc-0f44-4dda-aa64-2ef6b78515f7 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.857113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:54.881792 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1e65f-f935-4bd0-ad67-f4b9623ea88d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:54.886846 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.887518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.887806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.888091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.899213 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.900952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.906021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:54.906674 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:54.906974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:54.907242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:54.909271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:54.911842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1303/2599] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:54.973520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:54.974970 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:54.984617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1297/2600] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:55.277003 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:55.278661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:55.302002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337fca-4147-499e-94c8-6d0dd50c7720 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:55.308283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:55.309282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:55.309636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:55.309984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] PATH_INFO: `/v3/projects/9227eb3062d9431c91fd5a1bb7dc93d7` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:55.311244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9227eb3062d9431c91fd5a1bb7dc93d7)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:55.313328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:55.366156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9227eb3062d9431c91fd5a1bb7dc93d7', 'request_id': 'req-22821807-c561-49bd-a0e5-81fb5c7e3a7d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.372332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9227eb3062d9431c91fd5a1bb7dc93d7', 'request_id': 'req-22821807-c561-49bd-a0e5-81fb5c7e3a7d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.380475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9227eb3062d9431c91fd5a1bb7dc93d7', 'request_id': 'req-22821807-c561-49bd-a0e5-81fb5c7e3a7d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.394096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] The token cache is being invalidate because project 9227eb3062d9431c91fd5a1bb7dc93d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:55.394372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.395011 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.396592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1304/2601] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:55 2022] DELETE /identity/v3/projects/9227eb3062d9431c91fd5a1bb7dc93d7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:55.400720 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:55.412351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b6c1-d587-4aef-b936-6c2c1cd2a255 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:55.469067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:55.469936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:55.470867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:55.471216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] PATH_INFO: `/v3/users/42b8f108064b4124b81ea743ec0ce084` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:55.472403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b8f108064b4124b81ea743ec0ce084)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:55.474349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:55.519966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b8f108064b4124b81ea743ec0ce084'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.526744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b8f108064b4124b81ea743ec0ce084'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.527402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b8f108064b4124b81ea743ec0ce084'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.538181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:55.540684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b8f108064b4124b81ea743ec0ce084'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.548337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3769129e-22e8-48d5-89bd-9e874df7fa51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b8f108064b4124b81ea743ec0ce084'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.554461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1298/2602] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:55 2022] DELETE /identity/v3/users/42b8f108064b4124b81ea743ec0ce084 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:55.821125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:55.822743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:55.838315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22821807-c561-49bd-a0e5-81fb5c7e3a7d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:55.857818 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:55.898658 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:55.899453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:55.899744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:55.900025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] PATH_INFO: `/v3/projects/d34f840e1f684231952671021a34a288` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:55.901246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34f840e1f684231952671021a34a288)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:55.903185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:55.950135 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34f840e1f684231952671021a34a288', 'request_id': 'req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.956998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34f840e1f684231952671021a34a288', 'request_id': 'req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.964181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34f840e1f684231952671021a34a288', 'request_id': 'req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.975163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] The token cache is being invalidate because project d34f840e1f684231952671021a34a288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:55.975454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.976075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-860ac0fd-9aa7-4b7e-b8ca-dad4d7619ae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:55.977672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1305/2603] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:55 2022] DELETE /identity/v3/projects/d34f840e1f684231952671021a34a288 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:55.982518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:55.983034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:55.983142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.002644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:56.003853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:56.061938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1299/2604] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:56.071511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.071830 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.072145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.092258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:56.093875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:56.102850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1306/2605] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:56.107348 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.118981 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95771842-4e3d-4f9f-a9e0-2cffbf706375 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.123797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.124470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.124778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.125054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.125785 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:56.126095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.128101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.141242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1300/2606] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:56.146321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.156137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56d7fa-1ee3-482d-889b-f7ec1c917725 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.160946 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.161611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.161898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.162178 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.163035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.165013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.193875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1307/2607] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:56.198775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.208843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a2ffc-2435-47da-915b-0797bda7c120 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.277100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.277773 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.278061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.278341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.279063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.281020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.302550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1301/2608] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:56.304882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.316141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea56415-dd4c-4e32-a55f-6ff2d80f0e75 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.320929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.321514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.321800 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.322078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.322823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.324417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.337760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1308/2609] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:56.339754 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.351019 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47bbb7-0423-4f07-8df6-030e4b7675d6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.356632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.357007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.357296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.357574 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.358304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.359913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcc573-6772-4d41-9985-5fe56de93af9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.371540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1302/2610] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:56.375568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.390777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687a2b7-6b6e-40ea-a6cb-99ce6bb384a2 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.395798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.396719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.397005 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.397287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] PATH_INFO: `/v3/projects/589f1da4e8a0496a8763fe14b361c772/users/85151bc3c6c64f82a6cffe0349116430/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.417518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589f1da4e8a0496a8763fe14b361c772, user_id=85151bc3c6c64f82a6cffe0349116430, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.420526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.434095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1309/2611] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:56 2022] PUT /identity/v3/projects/589f1da4e8a0496a8763fe14b361c772/users/85151bc3c6c64f82a6cffe0349116430/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:56.444908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.445493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.445493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.479023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] MFA Rules not processed for user `85151bc3c6c64f82a6cffe0349116430`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:56.480249 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:56.525215 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.526156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:56.527063 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:56.551669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1303/2612] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:56.556226 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.583971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:56.587851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397696e4-d938-4f70-aa16-19451f641cc5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.594890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.595826 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.596286 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.597283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.599743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:56.620588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4363-b112-410b-8e92-975d6abce1ed None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.625408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.626067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.626878 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.627161 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.627847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.630002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.655293 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1304/2613] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:56.660054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.661396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.662859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.672267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1310/2614] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:56.724256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab4d2d-8668-4505-8afa-4594d27490bc admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.729100 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.729780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.730067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.730345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.731064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.733006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.754551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1305/2615] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:56.758817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.770724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7242c2c-3891-435d-824e-e902159ed495 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.775447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.776244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.776554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.776835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.777558 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.779481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.794114 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1311/2616] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:56.795264 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.806845 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a07222-f1cc-498d-b254-ab91c9844615 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.811490 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.812361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.812669 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.812941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] PATH_INFO: `/v3/projects/2e9ac03a4625414b85b0844bf3bcd189/users/10cc3be6db7d432d9d1b98ac82b215e7/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.830289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9ac03a4625414b85b0844bf3bcd189, user_id=10cc3be6db7d432d9d1b98ac82b215e7, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.833429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.847820 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1306/2617] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:56 2022] PUT /identity/v3/projects/2e9ac03a4625414b85b0844bf3bcd189/users/10cc3be6db7d432d9d1b98ac82b215e7/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:56.852030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.863129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ba8e-42c0-4e52-8005-8d419d1f184f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.926077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.926669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.926951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.927226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.927952 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.929591 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff954e8-d7bc-44f0-9dbe-a4257d141ff1 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.945325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1312/2618] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:56.946064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:56.957273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b565f2-c0d2-42de-8e3c-3b80e7b7dab8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:56.962093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:56.963021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.963305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.963583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] PATH_INFO: `/v3/projects/2e9ac03a4625414b85b0844bf3bcd189/users/10cc3be6db7d432d9d1b98ac82b215e7/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:56.965636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9ac03a4625414b85b0844bf3bcd189, user_id=10cc3be6db7d432d9d1b98ac82b215e7, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:56.968717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:56.979635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1307/2619] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:56 2022] PUT /identity/v3/projects/2e9ac03a4625414b85b0844bf3bcd189/users/10cc3be6db7d432d9d1b98ac82b215e7/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:56.990732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:56.991029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:56.991362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.021008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] MFA Rules not processed for user `10cc3be6db7d432d9d1b98ac82b215e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:57.022341 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:57.105075 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1313/2620] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:57.116245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:57.118705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:57.143336 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:57.180692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d76a7-a9f9-4265-90b7-b9a0831ab6e9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:57.184408 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:57.185081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:57.185360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:57.185666 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.195874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:57.197337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:57.209213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1308/2621] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:57.277026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:57.287384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73444b-f153-4f2e-8d6e-96f235c8e345 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:57.292060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:57.292769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:57.293054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:57.293332 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.295376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:57.306878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:57.381198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:57.382752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553123b4-5da2-4a20-bd37-44bcde4baf52 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:57.400874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1314/2622] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:57.594193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:57.657415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dd508-2509-41d0-9928-165fd672cf38 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:57.662269 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:57.663166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:57.663445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:57.663717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] PATH_INFO: `/v3/users/85151bc3c6c64f82a6cffe0349116430` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.664923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85151bc3c6c64f82a6cffe0349116430)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:57.666834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:57.714196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85151bc3c6c64f82a6cffe0349116430'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:57.721807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85151bc3c6c64f82a6cffe0349116430'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:57.722487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85151bc3c6c64f82a6cffe0349116430'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:57.736234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85151bc3c6c64f82a6cffe0349116430'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:57.743202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85151bc3c6c64f82a6cffe0349116430'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:57.749520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1309/2623] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:57 2022] DELETE /identity/v3/users/85151bc3c6c64f82a6cffe0349116430 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:57.754891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:57.755171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:57.755459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.772230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:57.773465 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:57.836570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1315/2624] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:57.846715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:57.848123 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:57.894659 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:57.907042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5204b6fd-bf20-4a24-8a5d-4540ff8105c6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:57.911678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:57.912318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:57.912632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:57.912901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:57.922010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:57.924476 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:57.934133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1310/2625] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:58.198026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:58.199666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:58.212683 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a5f76-d6a5-4382-9899-716e9aee8208 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:58.217738 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:58.218908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:58.219193 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:58.219474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] PATH_INFO: `/v3/projects/589f1da4e8a0496a8763fe14b361c772` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:58.221519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589f1da4e8a0496a8763fe14b361c772)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:58.223553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:58.270702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589f1da4e8a0496a8763fe14b361c772', 'request_id': 'req-b2e8b837-07c6-455f-afca-4072b9f6ea88'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.277006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589f1da4e8a0496a8763fe14b361c772', 'request_id': 'req-b2e8b837-07c6-455f-afca-4072b9f6ea88'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.285016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589f1da4e8a0496a8763fe14b361c772', 'request_id': 'req-b2e8b837-07c6-455f-afca-4072b9f6ea88'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.295322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] The token cache is being invalidate because project 589f1da4e8a0496a8763fe14b361c772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:58.295609 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.296249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.297804 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1316/2626] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:58 2022] DELETE /identity/v3/projects/589f1da4e8a0496a8763fe14b361c772 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:58.301999 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:58.311496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebdff47-8c44-4b65-b078-e5c34f3982ee service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:58.368771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:58.369726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:58.370012 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:58.370290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] PATH_INFO: `/v3/users/10cc3be6db7d432d9d1b98ac82b215e7` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:58.371539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10cc3be6db7d432d9d1b98ac82b215e7)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:58.373950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:58.421592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10cc3be6db7d432d9d1b98ac82b215e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.428729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10cc3be6db7d432d9d1b98ac82b215e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.429411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10cc3be6db7d432d9d1b98ac82b215e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.441545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10cc3be6db7d432d9d1b98ac82b215e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.449350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e998eb0a-7cf8-4c72-bea7-b71570acf4e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10cc3be6db7d432d9d1b98ac82b215e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:58.456664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1311/2627] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:18:58 2022] DELETE /identity/v3/users/10cc3be6db7d432d9d1b98ac82b215e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:59.129760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.131303 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.145586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b837-07c6-455f-afca-4072b9f6ea88 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.178018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:59.214719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.215698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.216048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.216330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] PATH_INFO: `/v3/projects/2e9ac03a4625414b85b0844bf3bcd189` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.217831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9ac03a4625414b85b0844bf3bcd189)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.220753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.277152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9ac03a4625414b85b0844bf3bcd189', 'request_id': 'req-659887e3-dd1f-4c99-afc7-0184731e9663'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:59.283933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9ac03a4625414b85b0844bf3bcd189', 'request_id': 'req-659887e3-dd1f-4c99-afc7-0184731e9663'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:59.291769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9ac03a4625414b85b0844bf3bcd189', 'request_id': 'req-659887e3-dd1f-4c99-afc7-0184731e9663'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:59.303878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] The token cache is being invalidate because project 2e9ac03a4625414b85b0844bf3bcd189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:59.304171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:59.304883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-659887e3-dd1f-4c99-afc7-0184731e9663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:59.306463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1317/2628] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:18:59 2022] DELETE /identity/v3/projects/2e9ac03a4625414b85b0844bf3bcd189 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:59.312094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.312599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.312722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.331234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:59.332430 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.398003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1312/2629] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:59.406019 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.406326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.406615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.428226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:59.429570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.437954 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1318/2630] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:59.442616 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.448767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:59.453632 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af087ea-c5e7-4c5d-8688-f4c24ccf1a8e None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.460204 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.461447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.461965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.462390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.463167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:59.463620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.465371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.479110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1313/2631] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:59.483362 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.493407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553597e-baf0-4799-aa8c-3c5f3ea528c2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.498353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.499472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.500060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.500587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.501892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.503893 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5fdbbc-80a1-4891-a6ac-6dbaebb9fb12 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.531628 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1319/2632] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:59.536780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.547690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d6958-b3a8-450c-be7b-d1fc0c713eb6 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.553708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.553995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.554283 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.572530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:59.573667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.609432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.610104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.610381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.610851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.611969 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.613933 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.630367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1320/2633] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:59.635201 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1314/2634] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:59.640349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.653436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267216fa-50eb-4e63-bb7d-eff36efdcf69 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.658724 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.659476 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.659764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.660038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.660797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.662406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.677069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1321/2635] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:59.678391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.678764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.680341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.691493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32860a-2986-4a45-b9ae-ed5cf7b4da1d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.696712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.697383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.697679 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.698025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.698761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.700379 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.711860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1322/2636] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:59.715737 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.726592 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2bf92-e283-45fe-bf9c-0f5590134729 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.731505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.732398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.732720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.733001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] PATH_INFO: `/v3/projects/862e2814149b4bce94f934b7e467a141/users/b64185fbb1cf4118b5ca5b9b3941705e/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.753499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:59.758534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=862e2814149b4bce94f934b7e467a141, user_id=b64185fbb1cf4118b5ca5b9b3941705e, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.762860 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3817b-2775-4c50-9fcc-1bf1ea042dbb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.780225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1323/2637] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:18:59 2022] PUT /identity/v3/projects/862e2814149b4bce94f934b7e467a141/users/b64185fbb1cf4118b5ca5b9b3941705e/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:59.785316 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375ea3b-1b88-465d-bab3-1b24a1435318 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:59.790833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.791559 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.792305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.824532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] MFA Rules not processed for user `b64185fbb1cf4118b5ca5b9b3941705e`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:59.827972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.862664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:59.863362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:59.863646 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:59.863930 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:59.892735 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:59.908262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1324/2638] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:59.922143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:59.925287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:59.935502 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:59.937174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:59.947523 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1315/2639] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:18:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:19:00.051570 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7f1d7-8f92-4235-9202-0780875ee54d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:00.057017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:00.057976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:00.058619 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:00.059116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:00.073372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:00.075289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e81b-c4d4-4791-a9f0-04781ee1acc4 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:00.100006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1325/2640] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:19:00.123479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:00.276398 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb448c-d515-47ff-ac77-34751f1e642e service swift] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:00.285991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:00.288136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:00.288555 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:00.288937 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] PATH_INFO: `/v3/users/b64185fbb1cf4118b5ca5b9b3941705e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:00.290914 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64185fbb1cf4118b5ca5b9b3941705e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:00.294765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:00.366847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64185fbb1cf4118b5ca5b9b3941705e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:00.377025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64185fbb1cf4118b5ca5b9b3941705e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:00.384890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64185fbb1cf4118b5ca5b9b3941705e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:00.404571 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64185fbb1cf4118b5ca5b9b3941705e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:00.415011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64185fbb1cf4118b5ca5b9b3941705e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:00.431357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1316/2641] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:19:00 2022] DELETE /identity/v3/users/b64185fbb1cf4118b5ca5b9b3941705e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:00.435971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:00.436494 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:00.436599 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:00.468908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:00.470171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:00.606799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1326/2642] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:00.625391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:00.628321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:00.692688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:00.737797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de609f65-56fe-458c-91ae-f9caddb9374d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:00.742499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:00.743323 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:00.743681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:00.744070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:00.760609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:00.762130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aaa9f3-3c79-4819-b937-09a25a4d3e23 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:00.774967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1317/2643] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:01.107029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.108566 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.121812 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3868a45-d1b3-4660-af1b-d7391e1c8df2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.129452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.129452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.129452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.129452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] PATH_INFO: `/v3/projects/862e2814149b4bce94f934b7e467a141` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.131742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862e2814149b4bce94f934b7e467a141)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.133751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.166103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:01.214805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862e2814149b4bce94f934b7e467a141', 'request_id': 'req-485c8e1e-4164-426a-b989-982dd86792b9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.230389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862e2814149b4bce94f934b7e467a141', 'request_id': 'req-485c8e1e-4164-426a-b989-982dd86792b9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.251008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862e2814149b4bce94f934b7e467a141', 'request_id': 'req-485c8e1e-4164-426a-b989-982dd86792b9'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.274329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] The token cache is being invalidate because project 862e2814149b4bce94f934b7e467a141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:19:01.275271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.276655 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-485c8e1e-4164-426a-b989-982dd86792b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.278577 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1327/2644] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:19:01 2022] DELETE /identity/v3/projects/862e2814149b4bce94f934b7e467a141 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:01.284421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.284987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.285338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.308253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:01.309610 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.378682 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1318/2645] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:01.387604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.387951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.388176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.410467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:01.411685 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.424582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1328/2646] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:01.425262 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.436385 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a5eaa-842f-4781-8226-4cdc41745ff2 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.445542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.446217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.446495 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.446769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.447501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:19:01.447810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.449463 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.465178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1319/2647] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:19:01.470045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.480289 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740820f-8082-4932-ac4e-1c8ca9ddcb48 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.485358 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.486742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.487028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.487302 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.488004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.489958 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.519762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1329/2648] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:19:01.523287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.534311 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d705532-b512-433f-8828-78524181eed2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.610547 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.613328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.629241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.629932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.630236 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.630499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.631243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.633239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.672332 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1320/2649] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:01.677166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.690540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac034b1-a9cb-426b-87d3-5f0f9ac1b6e8 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.700099 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.701250 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.701780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.702120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.703101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.705880 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.706060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0cc7b5-4928-4ff4-a8e6-9de7883a36ec admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.711335 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.712223 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.712537 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.712817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] PATH_INFO: `/v3/users/cbb63f6d0d104c21958bd6435bfa8cf2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.714049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb63f6d0d104c21958bd6435bfa8cf2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.716088 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.720518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1321/2650] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:19:01.725002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.737825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5cb58-c350-4d63-bf94-acd23df5c79d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.745010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.745730 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.746021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.746453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.747227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.748912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.761677 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1322/2651] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:01.766307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.782339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed673f-ef13-494f-98e2-f9566fed76e3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.787972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb63f6d0d104c21958bd6435bfa8cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.790360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.791260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.791541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.791881 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] PATH_INFO: `/v3/projects/c7677b551eae435caad1cc2e561af91a/users/130df5f0759c481d97abcd203a040702/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.799192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb63f6d0d104c21958bd6435bfa8cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.799934 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb63f6d0d104c21958bd6435bfa8cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.813148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7677b551eae435caad1cc2e561af91a, user_id=130df5f0759c481d97abcd203a040702, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.814668 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb63f6d0d104c21958bd6435bfa8cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.815611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-383184a5-4faa-424a-bd57-af99a892b466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb63f6d0d104c21958bd6435bfa8cf2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:01.816797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e542c6-7117-46ce-8dc2-9d62d569c605 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.828190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:01.834922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1330/2652] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:19:01 2022] DELETE /identity/v3/users/cbb63f6d0d104c21958bd6435bfa8cf2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:19:01.839633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.840306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.841134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.847384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1323/2653] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:19:01 2022] PUT /identity/v3/projects/c7677b551eae435caad1cc2e561af91a/users/130df5f0759c481d97abcd203a040702/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:19:01.848998 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.849304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.849582 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.864957 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:01.866119 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.879577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] MFA Rules not processed for user `130df5f0759c481d97abcd203a040702`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:01.880829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.886372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:01.956147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1331/2654] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:01.972519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.979112 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1324/2655] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:01.980137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:01.994724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.996823 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:02.077328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b1bba-d9b1-4111-9e19-9f0b62aa9159 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:02.082305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:02.082965 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:02.083243 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:02.083517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:02.094163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3bc3f-3ee9-4c68-a807-8c0fdd727da5 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:02.098839 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:02.100528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:02.100911 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:02.101667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:02.102025 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:02.102385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:02.113879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1325/2656] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:02.116438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:02.118029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:02.131414 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1332/2657] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:02.614694 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:02.616308 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:02.643472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f2d54-085d-4c6a-b6ca-d248b5bf28ff service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:02.648219 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:02.649134 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:02.649421 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:02.649699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] PATH_INFO: `/v3/projects/770da3da46e742de95a26d1ce0be6f0a` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:02.651419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=770da3da46e742de95a26d1ce0be6f0a)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:02.653709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:02.718525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770da3da46e742de95a26d1ce0be6f0a', 'request_id': 'req-e543c5a6-0333-4d8e-abac-49d68f9089e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.725548 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770da3da46e742de95a26d1ce0be6f0a', 'request_id': 'req-e543c5a6-0333-4d8e-abac-49d68f9089e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.734774 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770da3da46e742de95a26d1ce0be6f0a', 'request_id': 'req-e543c5a6-0333-4d8e-abac-49d68f9089e5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.752479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] The token cache is being invalidate because project 770da3da46e742de95a26d1ce0be6f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:19:02.752927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.753688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.755524 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1326/2658] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:19:02 2022] DELETE /identity/v3/projects/770da3da46e742de95a26d1ce0be6f0a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:02.759343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:02.772199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e9b5a-cc1a-40e8-990c-952d4052375c service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:02.839835 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:02.840746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:02.841036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:02.841314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] PATH_INFO: `/v3/users/aa44368cde984ec280692ac34a57a7af` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:02.842567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa44368cde984ec280692ac34a57a7af)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:02.844838 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:02.903282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:02.913014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa44368cde984ec280692ac34a57a7af'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.921059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa44368cde984ec280692ac34a57a7af'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.921711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa44368cde984ec280692ac34a57a7af'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.938635 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa44368cde984ec280692ac34a57a7af'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.940238 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa44368cde984ec280692ac34a57a7af'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:02.956308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1333/2659] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:19:02 2022] DELETE /identity/v3/users/aa44368cde984ec280692ac34a57a7af => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:03.264680 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:03.266090 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:03.289304 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c5a6-0333-4d8e-abac-49d68f9089e5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:03.329987 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:03.362579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:03.363464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:03.363747 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:03.364024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] PATH_INFO: `/v3/projects/e65a963e87a243088d186a3446dd2e6c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:03.365603 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65a963e87a243088d186a3446dd2e6c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:03.367625 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:03.421377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65a963e87a243088d186a3446dd2e6c', 'request_id': 'req-a799fe4a-3228-42f7-bf68-2c5036357e9e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.428224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65a963e87a243088d186a3446dd2e6c', 'request_id': 'req-a799fe4a-3228-42f7-bf68-2c5036357e9e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.434596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65a963e87a243088d186a3446dd2e6c', 'request_id': 'req-a799fe4a-3228-42f7-bf68-2c5036357e9e'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.466717 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] The token cache is being invalidate because project e65a963e87a243088d186a3446dd2e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:19:03.467069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.468600 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.470224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1327/2660] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:19:03 2022] DELETE /identity/v3/projects/e65a963e87a243088d186a3446dd2e6c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:03.475195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:03.478057 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:03.495999 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2538ffec-399d-4f92-a475-eae735a18dd7 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:03.565829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:03.566719 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:03.567006 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:03.567287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] PATH_INFO: `/v3/users/f514db5c26ff40088acba62205df9ce2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:03.568554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f514db5c26ff40088acba62205df9ce2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:03.570554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:03.651868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f514db5c26ff40088acba62205df9ce2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.660556 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f514db5c26ff40088acba62205df9ce2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.661173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f514db5c26ff40088acba62205df9ce2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.704522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f514db5c26ff40088acba62205df9ce2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.705879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-be8e3569-d22a-40f5-b2ba-4d19a2095a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f514db5c26ff40088acba62205df9ce2'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:03.725024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1334/2661] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:19:03 2022] DELETE /identity/v3/users/f514db5c26ff40088acba62205df9ce2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:04.153180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.155109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:04.173556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799fe4a-3228-42f7-bf68-2c5036357e9e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.250873 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.252159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.253180 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.253464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] PATH_INFO: `/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.254700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd2f3ec019a43adbe1560a2bd0031ac)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.256756 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.318241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd2f3ec019a43adbe1560a2bd0031ac', 'request_id': 'req-617e833a-39b9-462c-9468-5020d0d64401'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:04.325363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd2f3ec019a43adbe1560a2bd0031ac', 'request_id': 'req-617e833a-39b9-462c-9468-5020d0d64401'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:04.331614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd2f3ec019a43adbe1560a2bd0031ac', 'request_id': 'req-617e833a-39b9-462c-9468-5020d0d64401'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:04.334687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:04.349171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] The token cache is being invalidate because project 7bd2f3ec019a43adbe1560a2bd0031ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:19:04.350118 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:04.351004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-617e833a-39b9-462c-9468-5020d0d64401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:04.352703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1328/2662] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:19:04 2022] DELETE /identity/v3/projects/7bd2f3ec019a43adbe1560a2bd0031ac => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:04.357614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.358095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.358217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.382601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:04.384214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:04.430722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:04.464882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1335/2663] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:04.470980 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.471265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.471545 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.491739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:04.492988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:04.502744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1329/2664] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:04.507012 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.517581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d88e5-e8c7-4754-aede-37c79c6cd03d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.522565 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.523253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.523541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.523823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.524604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:19:04.524923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.526596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.543102 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1336/2665] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:19:04.544193 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.555148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd05719-329b-4890-8728-f36b3c4329d5 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.560185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.560935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.561224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.561501 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.562209 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.564926 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.601033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1330/2666] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:19:04 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:19:04.605101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.620069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5e749-1ccf-40e7-8c99-9c6cd9ea7c5b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.691923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.693712 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.694927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.695758 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.696654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.698744 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.724117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1337/2667] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:19:04 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:04.729178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.740054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b9ec4-79f5-4359-bd32-26c5ec4c0713 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.745702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.746399 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.746683 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.746960 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.747715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.749366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.765104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1331/2668] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:19:04.766440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.779292 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002b245-81a2-42e0-ab05-b7b1ca36b31b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.784524 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.785196 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.785482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.785760 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.786489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.788233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63368c0-8c5a-4a7f-9463-ae3c24a33674 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.802514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1338/2669] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:04.803962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:04.816876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ece1cc-39e9-4b9c-ad3c-8b110885270e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:04.821672 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:04.822554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.822836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.823107 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] PATH_INFO: `/v3/projects/ca5780630fe8464a99e6afe51c9bd3e0/users/4555cb3933ec45e687a72632857be7dd/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.841426 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5780630fe8464a99e6afe51c9bd3e0, user_id=4555cb3933ec45e687a72632857be7dd, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:04.844393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:04.861047 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1332/2670] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:19:04 2022] PUT /identity/v3/projects/ca5780630fe8464a99e6afe51c9bd3e0/users/4555cb3933ec45e687a72632857be7dd/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:04.865836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e98bb-7463-407e-b729-2fd4a533e3f8 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:04.866199 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e98bb-7463-407e-b729-2fd4a533e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:04.866557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e98bb-7463-407e-b729-2fd4a533e3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:04.885821 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-1c9e98bb-7463-407e-b729-2fd4a533e3f8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:04.887189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-1c9e98bb-7463-407e-b729-2fd4a533e3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:05.018639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1339/2671] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:05.028207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:05.030003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:05.101010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2cc41-a1e9-4a81-9a9c-569270e768f3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:05.105848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:05.106709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:05.107150 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:05.107601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:05.125418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:05.127637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:05.148596 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1333/2672] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:05.885375 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365107f-397f-4561-93bf-2461b7ebf803 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:05.885831 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365107f-397f-4561-93bf-2461b7ebf803 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:05.885935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365107f-397f-4561-93bf-2461b7ebf803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:05.924078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5365107f-397f-4561-93bf-2461b7ebf803 None None] MFA Rules not processed for user `4555cb3933ec45e687a72632857be7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:05.925279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5365107f-397f-4561-93bf-2461b7ebf803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:05.974016 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:06.026111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1340/2673] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:06.037126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:06.038647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:06.116076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67167d-5072-47b9-a38e-e223d1703274 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:06.124227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:06.125199 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:06.125598 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:06.125978 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:06.136504 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:06.139614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:06.150214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1334/2674] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:08.741650 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536fbda-4cba-4c1a-9ea0-ea4b1bd36c23 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:08.742176 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536fbda-4cba-4c1a-9ea0-ea4b1bd36c23 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:08.742277 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536fbda-4cba-4c1a-9ea0-ea4b1bd36c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:08.769304 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-9536fbda-4cba-4c1a-9ea0-ea4b1bd36c23 None None] MFA Rules not processed for user `4555cb3933ec45e687a72632857be7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:08.770550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9536fbda-4cba-4c1a-9ea0-ea4b1bd36c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:08.778815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1341/2675] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:11.040122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:11.041599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:11.056900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89664a1-2204-4d77-a13b-412aed0f4d49 service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:11.063196 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:11.064263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:11.065129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:11.065481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] PATH_INFO: `/v3/users/130df5f0759c481d97abcd203a040702` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:11.067224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] RBAC: Authorizing `identity:delete_user(user_id=130df5f0759c481d97abcd203a040702)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:11.068886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:11.112424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:11.132425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130df5f0759c481d97abcd203a040702'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:11.139857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130df5f0759c481d97abcd203a040702'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:11.140757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130df5f0759c481d97abcd203a040702'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:11.158335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130df5f0759c481d97abcd203a040702'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:11.181819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130df5f0759c481d97abcd203a040702'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:11.189343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1335/2676] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:19:11 2022] DELETE /identity/v3/users/130df5f0759c481d97abcd203a040702 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:11.194857 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966c717-902f-4977-9565-d70235fa556b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:11.196861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966c717-902f-4977-9565-d70235fa556b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:11.197153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966c717-902f-4977-9565-d70235fa556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:11.218493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-c966c717-902f-4977-9565-d70235fa556b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:11.220859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c966c717-902f-4977-9565-d70235fa556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:11.291810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1342/2677] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:11.315859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:11.317534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:11.391087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438fc6-296a-461e-9747-2cd57211932b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:11.395835 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:11.396620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:11.397321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:11.397321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:11.413804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:11.413804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a5291-afa6-4447-9a19-c877de4fc4d9 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:11.420876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1336/2678] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:12.027015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966c717-902f-4977-9565-d70235fa556b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.029185 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-c966c717-902f-4977-9565-d70235fa556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:12.042264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966c717-902f-4977-9565-d70235fa556b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.049947 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.050892 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.051182 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.051459 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] PATH_INFO: `/v3/projects/c7677b551eae435caad1cc2e561af91a` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.053621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7677b551eae435caad1cc2e561af91a)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.055611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.106916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7677b551eae435caad1cc2e561af91a', 'request_id': 'req-6a703620-d7f3-454e-a022-fde9d6699f2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:12.114490 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7677b551eae435caad1cc2e561af91a', 'request_id': 'req-6a703620-d7f3-454e-a022-fde9d6699f2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:12.122427 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7677b551eae435caad1cc2e561af91a', 'request_id': 'req-6a703620-d7f3-454e-a022-fde9d6699f2a'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:12.133230 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] The token cache is being invalidate because project c7677b551eae435caad1cc2e561af91a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:19:12.133529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:12.134150 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6a703620-d7f3-454e-a022-fde9d6699f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:19:12.135695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1343/2679] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:19:12 2022] DELETE /identity/v3/projects/c7677b551eae435caad1cc2e561af91a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:12.140731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.141231 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.141324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.159165 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:12.160358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:12.219455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1337/2680] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:12.228026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.228534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.228706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.248195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:12.249455 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:12.257851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1344/2681] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:12.261997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.271593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851a440-ed0c-486f-b4d1-b81f56c59e7f None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.276149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.276828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.277114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.277405 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.278120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:19:12.278429 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.280050 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.292007 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1338/2682] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:19:12.296247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.306263 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9b545-551b-4ae0-a0bb-7ec64347005b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.310955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.311598 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.311883 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.312159 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.312880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.314884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.339467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1345/2683] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:19:12 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:12.343810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.354140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595d232-46c1-4315-b09e-37b43a5c3475 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.387792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:12.412591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.413256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.413539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.413813 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.414491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.416403 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.442592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1339/2684] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:19:12 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:12.444015 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.455654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bb36e-622e-4afe-b5f4-deee20fb69ed admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.460487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.461151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.461436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.461718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.462587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.464211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.476000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1346/2685] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:12.479894 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.490153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86169ccf-738f-49fc-9102-8d28cee75bc1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.495330 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.495535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.495848 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.496130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.496923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.498609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fe831-5bac-4ce6-a84e-e929e40b9ef7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.510953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1340/2686] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:12.515249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.526140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0cc8e-c9ca-4805-8ef4-1748ca8cb862 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.530862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.531743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.532030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.532933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] PATH_INFO: `/v3/projects/67116cdacaa846138f7835ac5f53b863/users/60ff7c679e9747898bbc44da1d885b36/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.544561 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:12.549638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67116cdacaa846138f7835ac5f53b863, user_id=60ff7c679e9747898bbc44da1d885b36, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.553167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.567014 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1347/2687] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:19:12 2022] PUT /identity/v3/projects/67116cdacaa846138f7835ac5f53b863/users/60ff7c679e9747898bbc44da1d885b36/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:12.572725 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30f65-79c3-45c0-90d9-a71c58a606c2 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.573025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30f65-79c3-45c0-90d9-a71c58a606c2 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.573312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b30f65-79c3-45c0-90d9-a71c58a606c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.592010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-78b30f65-79c3-45c0-90d9-a71c58a606c2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:12.593505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-78b30f65-79c3-45c0-90d9-a71c58a606c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:12.652377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1341/2688] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:12.662644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:12.664359 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:12.723478 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265a037-31fc-4d75-98aa-ab9464f99025 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:12.727212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:12.727851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:12.728134 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:12.728408 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:12.737579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:12.739024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23dcc4-6c59-4da3-bbc8-9eb264c497dc service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:12.748243 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1348/2689] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:15.975355 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:15.975915 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:15.976002 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:15.978057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1342/2690] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:19:15.984018 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142266-951a-48e3-823e-dee4c971b35b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:15.984642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142266-951a-48e3-823e-dee4c971b35b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:15.984642 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72142266-951a-48e3-823e-dee4c971b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:16.006977 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-72142266-951a-48e3-823e-dee4c971b35b None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:16.008378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-72142266-951a-48e3-823e-dee4c971b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:16.100820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1349/2691] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:19:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:16.115592 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:16.117479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:16.183833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294328b-8318-4642-97b5-28a0664f89d4 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:16.187591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:16.188158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:16.188619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:16.188976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:16.200064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:16.201657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:16.213369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1343/2692] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:19:19.740546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f888-d262-428a-a244-eb542694d449 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:19.741140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f888-d262-428a-a244-eb542694d449 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:19.741140 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f888-d262-428a-a244-eb542694d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:19.773687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e111f888-d262-428a-a244-eb542694d449 None None] MFA Rules not processed for user `60ff7c679e9747898bbc44da1d885b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:19.774915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e111f888-d262-428a-a244-eb542694d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:19.871924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1350/2693] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:19.877493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:19.893034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ae8e1-a0e6-4935-b819-f464a21014e6 service glance] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:19.897718 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:19.898387 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:19.898810 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:19.899281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:19.899990 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:19.902059 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:19.933254 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1344/2694] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:19:19 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:19.937596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f888-d262-428a-a244-eb542694d449 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:19.949066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f888-d262-428a-a244-eb542694d449 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:19.960990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:20.028859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.029557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.029990 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.030271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.030968 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.033168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.056388 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1351/2695] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:19:19 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:19:20.057938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:20.069044 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9eb2f-330f-4399-add2-ece50f997c6c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:20.073748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.074411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.074698 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.074984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.076913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.078536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.089470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1345/2696] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:20.095380 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:20.106052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4c58c-6ef6-41f5-9c0b-6d846d116da9 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:20.112056 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.113062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.113625 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.113993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] PATH_INFO: `/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88/users/8cb6ab6278444aa8875607ad1e5cf19f/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.132054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0da8d3c342f4479a49f5ec0fa361f88, user_id=8cb6ab6278444aa8875607ad1e5cf19f, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.134769 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecd369-8422-4124-ae4c-2c7d566b7311 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.152206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1352/2697] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:19:20 2022] PUT /identity/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88/users/8cb6ab6278444aa8875607ad1e5cf19f/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:19:20.153461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:20.164016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b2120-6780-4720-b2a9-d8d5fc55ec86 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:20.174338 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.174613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.174894 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.195861 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:20.197811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:20.200629 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:20.245460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.246296 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.246997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.247328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.249324 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.252069 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.266110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1346/2698] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:19:20.270335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:20.276268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1353/2699] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:19:20.287281 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a8b5-dc95-4e53-b455-684b40adf28a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:20.289477 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:20.291147 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:20.295607 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.296535 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.296825 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.297108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] PATH_INFO: `/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88/users/8cb6ab6278444aa8875607ad1e5cf19f/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.300005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0da8d3c342f4479a49f5ec0fa361f88, user_id=8cb6ab6278444aa8875607ad1e5cf19f, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.302699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b9270-a1a3-4813-b073-083bd40ccf34 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.314589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1347/2700] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:19:20 2022] PUT /identity/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88/users/8cb6ab6278444aa8875607ad1e5cf19f/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:19:20.379572 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a5d9b-2df4-421b-be70-6d0321ce4d6f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:20.409859 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:20.457141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:20.457899 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:20.458328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:20.458675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:20.542752 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:20.544799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:20.560336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1354/2701] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:19:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:19:28.370344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d57c7-d9af-48ae-b809-c3938f4e6634 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:28.370344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d57c7-d9af-48ae-b809-c3938f4e6634 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:28.370344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d57c7-d9af-48ae-b809-c3938f4e6634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:28.421056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-680d57c7-d9af-48ae-b809-c3938f4e6634 None None] MFA Rules not processed for user `8cb6ab6278444aa8875607ad1e5cf19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:28.423274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-680d57c7-d9af-48ae-b809-c3938f4e6634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:28.546152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1348/2702] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:28.560691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:28.562327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:28.667330 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc9e60-b244-4d79-b7e5-9fff5b427a6f service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:28.673661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:28.674313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:28.674605 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:28.674885 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:28.731798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:28.733452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b81ed-6dc1-4c71-abdf-fd7f4f13e115 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:28.743467 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1355/2703] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:33.658688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:33.659242 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:33.659366 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:33.676806 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] MFA Rules not processed for user `4555cb3933ec45e687a72632857be7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:33.678039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:33.735605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1349/2704] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:19:38.739317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:43.728780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:45.802177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24bdcc-a721-49f1-bc81-6b9dde9d52f4 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:45.802177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24bdcc-a721-49f1-bc81-6b9dde9d52f4 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:45.802177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24bdcc-a721-49f1-bc81-6b9dde9d52f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:45.821321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-df24bdcc-a721-49f1-bc81-6b9dde9d52f4 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:45.822584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-df24bdcc-a721-49f1-bc81-6b9dde9d52f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:45.831890 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1356/2705] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:45.848714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:45.877363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c74d09-eba3-46d1-a8a9-9d36a10c6443 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:45.882569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:45.883251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:45.883533 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:45.883808 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:45.893223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:45.894815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4acac3-be86-44fc-a991-767547f9cb5e service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:45.903350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1350/2706] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:19:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:19:45.927551 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:45.927842 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:45.928118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:45.949110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] MFA Rules not processed for user `60ff7c679e9747898bbc44da1d885b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:45.950325 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:45.958618 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1357/2707] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:19:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:19:53.119477 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:53.119963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:53.120063 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:53.138006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:53.139213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:53.147415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1351/2708] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:53.160750 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:53.172923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f9fbb-0850-4b7f-8e6e-fca6302b7871 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:53.177953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:53.178593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:53.178880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:53.179163 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:53.187300 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:53.188942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:53.197801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1358/2709] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:20:03.003743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:03.005237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:03.087035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dec6fe-5b5f-4b6b-a9ba-6106c571befb None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:03.094559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:03.096120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:03.096634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:03.097119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:03.099202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:03.112968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:03.114593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1d577-4131-4b7d-ae32-a9808ebea0cf service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:03.126557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1352/2710] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:03.194314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=12 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:06.647089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:06.722089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f20c3c-4b47-42fb-9bf2-95e59b67b038 service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:06.726181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:06.726844 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:06.727137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:06.727416 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:06.740671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:06.743222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:06.758239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1359/2711] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:07.823137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:07.823656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:07.823757 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:07.842739 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:07.844423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:07.852319 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1353/2712] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:07.867008 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:07.880895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b65cd1-fd27-4cb1-a812-91427aacb9ab service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:07.886173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:07.886819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:07.887130 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:07.887419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:07.896924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:07.898534 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:07.907306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1360/2713] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:20:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:20:09.883392 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:09.884912 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:09.956149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f095fa-7cc5-4999-97b4-e677a608e087 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:09.960624 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:09.961493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:09.961767 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:09.962051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] PATH_INFO: `/v3/users/4555cb3933ec45e687a72632857be7dd` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:09.963235 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4555cb3933ec45e687a72632857be7dd)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:09.965175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.012312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4555cb3933ec45e687a72632857be7dd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.019288 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4555cb3933ec45e687a72632857be7dd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.019950 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4555cb3933ec45e687a72632857be7dd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.027003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:10.033657 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4555cb3933ec45e687a72632857be7dd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.041650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-8f388661-0577-42e3-961f-5b7366f405b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4555cb3933ec45e687a72632857be7dd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.047693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1354/2714] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:20:09 2022] DELETE /identity/v3/users/4555cb3933ec45e687a72632857be7dd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:10.224457 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.235139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b271600-433a-401d-877d-d6d04cfbb509 service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.292707 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.293498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.293786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.294072 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] PATH_INFO: `/v3/projects/ca5780630fe8464a99e6afe51c9bd3e0` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.295262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5780630fe8464a99e6afe51c9bd3e0)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.297219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.341970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5780630fe8464a99e6afe51c9bd3e0', 'request_id': 'req-b039dbd1-9f1a-4582-b759-edd14202a474'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.349066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5780630fe8464a99e6afe51c9bd3e0', 'request_id': 'req-b039dbd1-9f1a-4582-b759-edd14202a474'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.355938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5780630fe8464a99e6afe51c9bd3e0', 'request_id': 'req-b039dbd1-9f1a-4582-b759-edd14202a474'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.366468 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] The token cache is being invalidate because project ca5780630fe8464a99e6afe51c9bd3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:10.366761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.367399 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-b039dbd1-9f1a-4582-b759-edd14202a474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:10.368966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1361/2715] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:20:10 2022] DELETE /identity/v3/projects/ca5780630fe8464a99e6afe51c9bd3e0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:20:10.373946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.374237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.374521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.394895 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:10.396116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:10.455268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1355/2716] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:10.462868 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.463156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.463454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.482391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:10.483718 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:10.491636 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1362/2717] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:10.495692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.505379 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e2b6-e2e4-4af0-b444-18f60773c7a1 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.509974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.510630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.510917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.511195 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.511919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:20:10.512230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.513879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.526040 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1356/2718] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:20:10.530456 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.540923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbfd44-08f7-4449-9be6-18a2291a509e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.545557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.546137 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.546417 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.546695 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.547369 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.549729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.559349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:10.578124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1363/2719] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:20:10 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:20:10.582879 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.593088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f940dfa-c64c-4c8c-a2d3-98147f88c787 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.650758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.651440 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.651731 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.652078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.652817 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.654746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.675042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1357/2720] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:20:10 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:20:10.679126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.690225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f200351-c5ea-41ae-a27c-11d07602799c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.694925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.695582 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.695870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.696149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.696979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.698593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.709568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1364/2721] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:20:10.713542 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.723977 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46302a-7442-482d-bd7e-d0963c679c83 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.728807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.729460 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.729744 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.730022 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.730736 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.732335 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c2aa2-465c-4b29-bc89-b42f723bd3c0 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.742370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1358/2722] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:20:10.746247 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.756828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9223c6-493c-4169-b261-aa8320eeffef admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.761487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.762354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.762640 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.762920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] PATH_INFO: `/v3/projects/f21113c02bc246ed96f23302f966a111/users/6279aefee879499bad5f43a45f27c984/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.778323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f21113c02bc246ed96f23302f966a111, user_id=6279aefee879499bad5f43a45f27c984, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.781152 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.793824 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1365/2723] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:20:10 2022] PUT /identity/v3/projects/f21113c02bc246ed96f23302f966a111/users/6279aefee879499bad5f43a45f27c984/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:10.803721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.804103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.804472 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.831826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] MFA Rules not processed for user `6279aefee879499bad5f43a45f27c984`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:10.833080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:10.869885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:10.897750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1359/2724] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:10.909235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:10.910912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:10.966959 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe7e2-50b7-47a5-830b-b1938e5ce91f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:10.971175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:10.971843 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:10.972406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:10.972714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:10.981050 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:10.982486 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:10.992314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1366/2725] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:20.894857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=10 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:20.988149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:27.495692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:27.497172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:27.581370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ebd51c-6c5d-4be3-8ba8-dea401207fd0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:27.586819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:27.587465 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:27.587748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:27.588030 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:27.589824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:27.651892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:27.653588 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:27.663130 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1360/2726] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:20:37.658433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=56 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:44.334189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:44.336334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:44.416903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df344f82-7a9f-431d-9e2b-4c0a01771636 service glance] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:44.421600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:44.422260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:44.422723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:44.423001 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:44.424888 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:44.492294 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:44.493805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a5af5-cdeb-4b95-8541-8ae02402e1b4 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:44.503639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1367/2727] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:20:45.933530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:45.945840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d21ef5-a4fe-401a-9ff1-77ef516603b3 service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:45.950771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:45.951941 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:45.952354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:45.952762 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] PATH_INFO: `/v3/users/6279aefee879499bad5f43a45f27c984` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:45.954159 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6279aefee879499bad5f43a45f27c984)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:45.956315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:46.013521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6279aefee879499bad5f43a45f27c984'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.022506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6279aefee879499bad5f43a45f27c984'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.023413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6279aefee879499bad5f43a45f27c984'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.040796 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6279aefee879499bad5f43a45f27c984'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.063843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6279aefee879499bad5f43a45f27c984'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.063843 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1361/2728] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:20:45 2022] DELETE /identity/v3/users/6279aefee879499bad5f43a45f27c984 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:46.067418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:46.068953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:46.068953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:46.097597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:46.099708 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:46.180029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1368/2729] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:46.193853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:46.196819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:46.304188 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e38d-c238-458d-baea-178e52ad7252 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:46.312095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064a86-d392-4818-b166-20383194d992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:46.315035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a86-d392-4818-b166-20383194d992 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:46.315655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a86-d392-4818-b166-20383194d992 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:46.316153 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a86-d392-4818-b166-20383194d992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:46.329285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064a86-d392-4818-b166-20383194d992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:46.331883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064a86-d392-4818-b166-20383194d992 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:46.350342 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1362/2730] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:46.920138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:46.922024 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:46.937762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4bae-9b43-4342-8298-7048ebbee57b None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:46.943469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:46.944343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:46.944653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:46.944925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] PATH_INFO: `/v3/projects/f21113c02bc246ed96f23302f966a111` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:46.946404 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f21113c02bc246ed96f23302f966a111)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:46.948340 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.004589 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:47.010454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f21113c02bc246ed96f23302f966a111', 'request_id': 'req-de59db3c-39a2-4909-862b-dbdd4bd6d55d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:47.017798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f21113c02bc246ed96f23302f966a111', 'request_id': 'req-de59db3c-39a2-4909-862b-dbdd4bd6d55d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:47.026333 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f21113c02bc246ed96f23302f966a111', 'request_id': 'req-de59db3c-39a2-4909-862b-dbdd4bd6d55d'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:47.041116 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] The token cache is being invalidate because project f21113c02bc246ed96f23302f966a111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:47.041412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:47.042995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-de59db3c-39a2-4909-862b-dbdd4bd6d55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:47.044836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1369/2731] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:20:46 2022] DELETE /identity/v3/projects/f21113c02bc246ed96f23302f966a111 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:47.050688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.051552 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.051834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.071191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:47.072327 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:47.195233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.197030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.197562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.201795 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1363/2732] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:47.222531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:47.224258 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:47.238959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.241144 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1370/2733] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:47.252151 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8e7f-1a88-4863-b20b-56c8e87062ce None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.265372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.266265 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.266568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.266852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.267605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80314) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:20:47.267919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.269752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.284352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1364/2734] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:20:47.288991 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.298763 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d6e7e-a6c2-4de0-878d-be6e18f19aed None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.305732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.306633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.307064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.307531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.308420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.310538 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.356248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1371/2735] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:20:47.360110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.366295 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:47.373367 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4566-cf4b-4201-9a17-2e9c7ccb8143 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.442190 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.443701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.444172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.444777 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] PATH_INFO: `/v3/users` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.445707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.448148 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.475917 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1365/2736] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:20:47.479681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.491432 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360644-ce2e-49be-8a00-e3b9d60af7b4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.500992 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.502324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.502759 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.505272 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.506913 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.509709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.530097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.532539 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1372/2737] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:20:47.541419 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e92fd-9d7b-4b7f-b210-5c6c0a461506 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.547240 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.548087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.548381 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.548902 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.549549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.551207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016a772-cdbb-4355-ae8c-c5209bd6605b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.580686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.583973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1366/2738] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:20:47.593013 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bf3c-eab0-4e33-bab8-6edb7815a409 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.598060 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.599084 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.599484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.603688 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] PATH_INFO: `/v3/projects/be77db463e844d0a8078b9fd07b7a968/users/2c0fe34091fa457ba6a20f6d0b9fb016/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.626529 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be77db463e844d0a8078b9fd07b7a968, user_id=2c0fe34091fa457ba6a20f6d0b9fb016, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.629453 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73739754-44d4-46ef-828c-69b83fb252c0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.656615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1373/2739] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:20:47 2022] PUT /identity/v3/projects/be77db463e844d0a8078b9fd07b7a968/users/2c0fe34091fa457ba6a20f6d0b9fb016/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:20:47.658857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.659260 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.659647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.688095 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:47.689863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:47.756749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70bd17-e6da-455e-bdce-febbeee0f572 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.757049 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70bd17-e6da-455e-bdce-febbeee0f572 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.757472 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70bd17-e6da-455e-bdce-febbeee0f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.783775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0a70bd17-e6da-455e-bdce-febbeee0f572 None None] MFA Rules not processed for user `973c068eff44436785ddf19a6465da8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:47.786041 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0a70bd17-e6da-455e-bdce-febbeee0f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:47.813781 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1367/2740] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:47.824253 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:47.826163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:47.880810 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1374/2741] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:47.923699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1300c-166b-4bb8-a863-2f9bc84cd5b8 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:47.929772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:47.931801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:47.932185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:47.932587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:47.948968 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:47.951263 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:47.970096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1368/2742] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:20:49.019218 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175973-365c-4273-8c6b-80f0acac1433 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:49.019780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175973-365c-4273-8c6b-80f0acac1433 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:49.019780 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8175973-365c-4273-8c6b-80f0acac1433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:49.050880 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e8175973-365c-4273-8c6b-80f0acac1433 None None] MFA Rules not processed for user `2c0fe34091fa457ba6a20f6d0b9fb016`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:49.053167 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e8175973-365c-4273-8c6b-80f0acac1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:49.137667 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1375/2743] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:49.145589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:49.147251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:49.182277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:49.231077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f96a5-bc51-4a25-b0ad-d363ad810b67 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:49.238481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:49.239901 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:49.240738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:49.241245 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:49.253116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:49.254922 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:49.268136 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1369/2744] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:20:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:50.773742 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:50.774201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:50.774501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:50.783039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:50.801862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] MFA Rules not processed for user `2c0fe34091fa457ba6a20f6d0b9fb016`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:50.805131 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:50.814791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1376/2745] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:52.945996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:52.947404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:52.965709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8a012-eafb-483a-aa0c-ae0b5c9a57fd service cinder] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:52.970830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:52.971778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:52.972117 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:52.972334 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] PATH_INFO: `/v3/users/3b7eb6c795964aaa92e7f1fd3993a29c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:52.973556 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7eb6c795964aaa92e7f1fd3993a29c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:52.975534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:53.026453 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7eb6c795964aaa92e7f1fd3993a29c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.036788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7eb6c795964aaa92e7f1fd3993a29c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.037854 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7eb6c795964aaa92e7f1fd3993a29c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.066223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7eb6c795964aaa92e7f1fd3993a29c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.078597 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7eb6c795964aaa92e7f1fd3993a29c'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.088684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1370/2746] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:20:52 2022] DELETE /identity/v3/users/3b7eb6c795964aaa92e7f1fd3993a29c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:53.447151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:53.479067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeeaf5f-cb50-4452-9036-4e45c7170e79 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:53.553920 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:53.554806 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:53.555089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:53.555364 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] PATH_INFO: `/v3/projects/32fb068c248a4cb9bb0e73dab923ca19` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:53.556622 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32fb068c248a4cb9bb0e73dab923ca19)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:53.558555 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:53.614113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32fb068c248a4cb9bb0e73dab923ca19', 'request_id': 'req-e7a4f39b-549e-49a3-aad9-6274d06ce407'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.624836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32fb068c248a4cb9bb0e73dab923ca19', 'request_id': 'req-e7a4f39b-549e-49a3-aad9-6274d06ce407'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.632601 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32fb068c248a4cb9bb0e73dab923ca19', 'request_id': 'req-e7a4f39b-549e-49a3-aad9-6274d06ce407'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.645172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] The token cache is being invalidate because project 32fb068c248a4cb9bb0e73dab923ca19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:53.645463 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.646434 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.648113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1377/2747] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:20:53 2022] DELETE /identity/v3/projects/32fb068c248a4cb9bb0e73dab923ca19 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:53.652092 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:53.653639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:53.668270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f1322-ee59-4be0-a61a-1f0e9371022d admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:53.728114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:53.729043 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:53.729331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:53.729612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] PATH_INFO: `/v3/users/82715aba02d6401b8789590e44292487` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:53.730838 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82715aba02d6401b8789590e44292487)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:53.732809 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:53.775033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:53.780911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82715aba02d6401b8789590e44292487'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.791362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82715aba02d6401b8789590e44292487'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.792282 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82715aba02d6401b8789590e44292487'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.807760 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82715aba02d6401b8789590e44292487'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.814764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-025ba2b3-8c43-41f1-983e-44d9d7d416a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82715aba02d6401b8789590e44292487'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:53.822338 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1371/2748] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:20:53 2022] DELETE /identity/v3/users/82715aba02d6401b8789590e44292487 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:54.003315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:54.004764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:54.018971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4f39b-549e-49a3-aad9-6274d06ce407 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:54.078382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:54.079264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:54.079546 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:54.079820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] PATH_INFO: `/v3/projects/a119c636d156473fb72d5fe4ec32d4da` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:54.081120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a119c636d156473fb72d5fe4ec32d4da)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:54.083833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:54.119070 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:54.135246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a119c636d156473fb72d5fe4ec32d4da', 'request_id': 'req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:54.141531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a119c636d156473fb72d5fe4ec32d4da', 'request_id': 'req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:54.149531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a119c636d156473fb72d5fe4ec32d4da', 'request_id': 'req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:54.160687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] The token cache is being invalidate because project a119c636d156473fb72d5fe4ec32d4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:54.163071 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:54.163756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:54.165616 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1378/2749] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:20:54 2022] DELETE /identity/v3/projects/a119c636d156473fb72d5fe4ec32d4da => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:03.818067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=14,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:04.157834 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:53.553065 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:53.553599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:53.553716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:53.571362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] MFA Rules not processed for user `2c0fe34091fa457ba6a20f6d0b9fb016`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:53.572599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:53.632182 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1372/2750] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:58.555925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:58.558125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:58.623037 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5e526-ca77-49bc-b20b-17b6c5011c2e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:58.626682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:58.627339 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:58.627626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:58.627907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:58.629746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:58.702995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:58.704552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882f63-7099-4e01-bf02-d8c8bdddacdf service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:58.716312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1379/2751] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:21:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:01.213911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:01.223802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18b674-b577-4917-b42b-8af2219deacd None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:01.227396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:01.228035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:01.228318 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:01.228622 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:01.230955 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:01.246170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:01.247834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:01.260439 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1373/2752] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:08.710649 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=54 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:11.254897 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=31 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:24.184255 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60ae1-e8b0-439c-a5d3-a3360442daa0 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:24.184790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60ae1-e8b0-439c-a5d3-a3360442daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:24.184875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f60ae1-e8b0-439c-a5d3-a3360442daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:24.202923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e6f60ae1-e8b0-439c-a5d3-a3360442daa0 None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:24.204173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e6f60ae1-e8b0-439c-a5d3-a3360442daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:24.263578 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1380/2753] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:22:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:24.311049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:24.312491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:24.397033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818e115-fc46-4c0c-8df2-974b0a82ecb7 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:24.402423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:24.403210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:24.403748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:24.404135 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:24.413074 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:24.414687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:24.423675 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1374/2754] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:22:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:26.819884 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:26.820376 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:26.820682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:26.839222 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] MFA Rules not processed for user `60ff7c679e9747898bbc44da1d885b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:26.841183 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:26.905873 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1381/2755] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:22:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:28.075701 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:28.077827 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:28.196380 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07a670-c93f-4420-8496-7c78bcbc2724 service swift] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:28.202528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:28.203467 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:28.203819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:28.206830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:28.209371 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:28.231273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:28.233010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:28.244361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1375/2756] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:31.534614 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:31.536499 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:31.555398 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81c02-b0c5-466b-a171-5b8378152731 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:31.561675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:31.567778 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:31.568403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:31.568721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:31.574956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:31.607268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:31.611264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:31.636908 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1382/2757] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:32.556128 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:32.558645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:32.580765 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6a2f2-b231-447a-955e-958f34fe0526 service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:32.594911 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:32.596527 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:32.597116 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:32.597569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:32.599850 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:32.632936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:32.634567 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce7f3f-db72-4d5b-9e76-2c683c7cc54c service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:32.649520 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1376/2758] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:41.617488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=56 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:42.642431 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=70 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:43.152516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:43.170438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171b632-31b6-4a2a-9e71-dd205ea290cc service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:43.175095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:43.175828 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.176284 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.176782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.178989 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:43.189448 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:43.190932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b8ec-4853-4dbc-a774-50707835ea4e service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:43.199048 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1383/2759] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:22:43.307451 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.307818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.308276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.309743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1377/2760] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:22:43.318714 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613f92-77ce-4b45-ba6b-c6058f74e521 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.319047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613f92-77ce-4b45-ba6b-c6058f74e521 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.319506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613f92-77ce-4b45-ba6b-c6058f74e521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.338097 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cf613f92-77ce-4b45-ba6b-c6058f74e521 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:43.339822 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cf613f92-77ce-4b45-ba6b-c6058f74e521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:43.350078 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1384/2761] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:43.363801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:43.365617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:43.432349 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e25ecc-d1b7-403e-aefa-c07df93c1282 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:43.435947 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36672cdeb2f148698b116e06095955d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:43.436630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.436920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.437200 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.447787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:43.449782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc572b02-296c-4440-bc50-993f5f55d6f3 service glance] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:43.460553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1378/2762] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:22:43.526430 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056c85-754d-4620-8f3e-2d3b558d27df None None] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.527541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056c85-754d-4620-8f3e-2d3b558d27df None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.528022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26056c85-754d-4620-8f3e-2d3b558d27df None None] PATH_INFO: `/` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.529443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1385/2763] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:22:43.534218 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9641-f7e8-42ce-8e3c-5d282cbe8a43 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.534505 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9641-f7e8-42ce-8e3c-5d282cbe8a43 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.534778 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf9641-f7e8-42ce-8e3c-5d282cbe8a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.575175 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-61cf9641-f7e8-42ce-8e3c-5d282cbe8a43 None None] MFA Rules not processed for user `9f3ecb0e259e4ed89f354e0e95927f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:43.576627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-61cf9641-f7e8-42ce-8e3c-5d282cbe8a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:43.583513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1379/2764] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:43.597652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056c85-754d-4620-8f3e-2d3b558d27df None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:43.617922 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26056c85-754d-4620-8f3e-2d3b558d27df None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:43.621721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:43.622396 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.622686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.622966 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.630307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:43.631753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3ede-cc8a-4d42-9e92-cdbe6e2b0d9b service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:43.640382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1386/2765] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:43.734057 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484e43-49e1-4954-ad19-519ef0bd6796 None None] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.734518 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484e43-49e1-4954-ad19-519ef0bd6796 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.734636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484e43-49e1-4954-ad19-519ef0bd6796 None None] PATH_INFO: `/` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.735967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1380/2766] 10.222.0.41 () {54 vars in 921 bytes} [Fri May 27 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:22:43.741576 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0222c-172e-462a-81ec-f164d6340366 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.741869 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0222c-172e-462a-81ec-f164d6340366 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.742151 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0222c-172e-462a-81ec-f164d6340366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.760740 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0ca0222c-172e-462a-81ec-f164d6340366 None None] MFA Rules not processed for user `f2d44da5ce124bd5b18462136f91fcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:43.762629 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0ca0222c-172e-462a-81ec-f164d6340366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:43.771921 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1387/2767] 10.222.0.41 () {58 vars in 1018 bytes} [Fri May 27 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:43.782583 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484e43-49e1-4954-ad19-519ef0bd6796 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:43.793699 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484e43-49e1-4954-ad19-519ef0bd6796 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:43.797454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:43.798089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:43.798369 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:43.798648 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:43.809145 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:43.810564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:43.821386 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1381/2768] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:22:48.733776 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.734241 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.734452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.754026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:48.755220 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:48.763820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1388/2769] 10.222.0.41 () {58 vars in 1009 bytes} [Fri May 27 15:22:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:48.776004 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:48.789068 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed5efa-3690-4396-a66f-9cac1371b9f8 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:48.795540 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:48.796086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.796389 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.796834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.805445 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:48.807023 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:48.817098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1382/2770] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:22:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:51.748659 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:51.750092 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:51.798317 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9277995-acb2-4d05-a682-b13976427f2c None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:51.811109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:51.812141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:51.812694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:51.813202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:51.818709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:51.844278 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:51.846516 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:51.861704 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1389/2771] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:54.180438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:54.195298 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503cbd3-9d79-4879-a34e-e328310c2f03 service swift] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:54.201397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:54.202070 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:54.202474 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:54.202761 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:54.204573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:54.269743 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:54.271498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:54.282274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1383/2772] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:57.417065 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:57.429932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df40b4-5ce9-4fbb-bcb9-e5f2db8dae91 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:57.437365 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:57.449694 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:57.450343 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:57.450960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:57.453522 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:57.466677 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:57.468428 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:57.484282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1390/2773] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:57.485534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:57.486954 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:57.562345 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:57.568711 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac0188-2a7b-4577-bf0d-a0119d5b3958 service nova] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:57.574270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:57.575322 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:57.575684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:57.576039 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] PATH_INFO: `/v3/users/2c0fe34091fa457ba6a20f6d0b9fb016` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:57.577481 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0fe34091fa457ba6a20f6d0b9fb016)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:57.580011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:57.634920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0fe34091fa457ba6a20f6d0b9fb016'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:57.643230 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0fe34091fa457ba6a20f6d0b9fb016'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:57.643946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0fe34091fa457ba6a20f6d0b9fb016'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:57.657991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0fe34091fa457ba6a20f6d0b9fb016'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:57.665892 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-50852ced-0680-4f27-ad16-59957f02171e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0fe34091fa457ba6a20f6d0b9fb016'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:57.677956 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1384/2774] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:22:57 2022] DELETE /identity/v3/users/2c0fe34091fa457ba6a20f6d0b9fb016 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:57.864324 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:57.877729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7ba59-dc8a-4a41-b7cd-aeae7640ea40 service placement] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:57.944268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:57.945232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:57.945794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:57.946172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] PATH_INFO: `/v3/projects/be77db463e844d0a8078b9fd07b7a968` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:57.947474 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=be77db463e844d0a8078b9fd07b7a968)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:57.950031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:58.004081 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be77db463e844d0a8078b9fd07b7a968', 'request_id': 'req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:58.010816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be77db463e844d0a8078b9fd07b7a968', 'request_id': 'req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:58.020871 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be77db463e844d0a8078b9fd07b7a968', 'request_id': 'req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:58.034919 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] The token cache is being invalidate because project be77db463e844d0a8078b9fd07b7a968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80315) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:58.035077 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:58.035747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:58.037532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1391/2775] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:22:57 2022] DELETE /identity/v3/projects/be77db463e844d0a8078b9fd07b7a968 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:04.386938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:04.387337 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:04.387519 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:04.406292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] MFA Rules not processed for user `fa4fbb1e67c24c2b8a1d816c59c75a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:23:04.407513 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:04.462418 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1385/2776] 10.222.0.41 () {58 vars in 967 bytes} [Fri May 27 15:23:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:23:04.475029 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:04.476419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:04.497798 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:04.548122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3833a8-c4c4-4b8c-ae7e-f2fb5b1cef9e admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:04.553053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b7e595d067a42dca56f0ef5fe8485c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:04.553696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:04.553975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:04.554248 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:04.562786 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:04.564372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:04.572246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1392/2777] 10.222.0.41 () {60 vars in 1384 bytes} [Fri May 27 15:23:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:23:14.455857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:14.569047 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=24 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:56.063058 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:56.064434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:56.170350 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9d98f-c6df-4f20-89d9-5f591020972c None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:56.174053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:56.174692 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:56.174971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:56.175244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:56.177016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:56.283755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:56.285178 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:56.304506 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1386/2778] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:59.362511 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:59.363906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:59.417181 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:59.418942 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:59.432166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bee4d0-e643-477a-b39b-6e8e32f06950 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:59.439938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:59.440853 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:59.441417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:59.441702 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] PATH_INFO: `/v3/users/973c068eff44436785ddf19a6465da8f` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:59.442994 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=973c068eff44436785ddf19a6465da8f)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:59.445020 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:59.453569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:59.501631 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973c068eff44436785ddf19a6465da8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.509216 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973c068eff44436785ddf19a6465da8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.509936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973c068eff44436785ddf19a6465da8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.510470 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ede382-5e22-45ab-9844-255a7dcdc63e service swift] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:59.517643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:59.518419 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:59.518713 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:59.518998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:59.520855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:59.525826 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973c068eff44436785ddf19a6465da8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.533676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:59.535390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0138f-2fbf-4ac6-9368-b072be53e640 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:59.536077 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973c068eff44436785ddf19a6465da8f'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.543415 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1387/2779] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:23:59 2022] DELETE /identity/v3/users/973c068eff44436785ddf19a6465da8f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:59.548929 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1393/2780] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:23:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:59.727232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:59.728706 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:59.797361 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2fee4-9015-4380-9eff-8b2a477242ca admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:59.801883 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:59.802732 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:59.803011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:59.803274 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] PATH_INFO: `/v3/projects/80d48e2adc1e4c6ebfdd43e84f8e3106` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:59.804464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d48e2adc1e4c6ebfdd43e84f8e3106)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:59.806374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:59.853483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d48e2adc1e4c6ebfdd43e84f8e3106', 'request_id': 'req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.859726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d48e2adc1e4c6ebfdd43e84f8e3106', 'request_id': 'req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.865232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d48e2adc1e4c6ebfdd43e84f8e3106', 'request_id': 'req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.878093 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] The token cache is being invalidate because project 80d48e2adc1e4c6ebfdd43e84f8e3106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:23:59.878384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.879011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-39cad85f-30b7-45f2-9bc1-bfbaa4676a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:59.880557 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1388/2781] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:23:59 2022] DELETE /identity/v3/projects/80d48e2adc1e4c6ebfdd43e84f8e3106 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:59.886126 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:59.886500 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:59.886852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:59.935125 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:23:59.936234 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:59.994734 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1394/2782] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:00.002713 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.003003 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.003283 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.021997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:00.023185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:00.031078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1389/2783] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:00.035086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.044855 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d61f12-f6a8-425d-8a78-699206f3bb4d None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.049389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.050046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.050329 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.050608 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.051327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:00.051637 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.053282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.065139 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1395/2784] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:00.069206 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.079256 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd848ec-f731-4777-a088-5aecf479d73d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.083928 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.084608 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.084896 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.085174 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.085871 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.087770 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.092498 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:00.117671 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1390/2785] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:00 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:24:00.122801 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.132782 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc147-e032-4101-abae-0eb61d36ed7b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.154035 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=100 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:00.190378 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.190961 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.191244 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.191519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.192201 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.194173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.214314 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1396/2786] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:00 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:00.218470 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.229096 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0014d-fd8b-4ce7-aa86-520c9a1409f4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.234579 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.235238 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.235525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.235807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.236560 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.238170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.249191 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1391/2787] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:00.253031 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.262845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b1a3e-145d-4a58-a74e-022a370257cb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.268815 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.269473 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.269757 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.270033 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.270802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.272411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38ca50-9947-4beb-b484-766256729fc0 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.282998 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1397/2788] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:00.287149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.298015 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15409b74-b0c4-4051-ad4c-4c459470ed25 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.302663 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.303564 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.303851 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.304133 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] PATH_INFO: `/v3/projects/940e041b80564c1f92090a3bd98a939e/users/a5149df620c64dc19694a26b66645e50/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.319362 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] RBAC: Authorizing `identity:create_grant(project_id=940e041b80564c1f92090a3bd98a939e, user_id=a5149df620c64dc19694a26b66645e50, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.322048 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.335536 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1392/2789] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:00 2022] PUT /identity/v3/projects/940e041b80564c1f92090a3bd98a939e/users/a5149df620c64dc19694a26b66645e50/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:00.346039 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.346334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.346615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.375829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] MFA Rules not processed for user `a5149df620c64dc19694a26b66645e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:00.377160 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:00.443264 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1398/2790] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:24:00.454011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:00.455729 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:00.521493 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292e3cd-c802-44eb-a66f-337c4836700b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:00.525110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:00.525746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:00.526202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:00.526719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:00.535088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:00.536537 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfb9ea-e3e2-4cd0-87a8-7003a9ce1f9e service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:00.546120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1393/2791] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:08.805368 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:08.869615 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc101db7-c85b-4403-a0ed-ec82a3ba7689 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:08.874711 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:08.875581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:08.875882 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:08.876143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] PATH_INFO: `/v3/users/a5149df620c64dc19694a26b66645e50` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:08.877353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5149df620c64dc19694a26b66645e50)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:08.879281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:08.915355 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:08.926239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5149df620c64dc19694a26b66645e50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:08.935040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5149df620c64dc19694a26b66645e50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:08.935653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5149df620c64dc19694a26b66645e50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:08.947586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5149df620c64dc19694a26b66645e50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:08.948793 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5149df620c64dc19694a26b66645e50'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:08.960942 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1399/2792] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:08 2022] DELETE /identity/v3/users/a5149df620c64dc19694a26b66645e50 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:08.965651 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:08.966137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:08.966255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:08.983852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:08.985094 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:09.045163 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1394/2793] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:09.054730 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.056249 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:09.115754 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc46a0-3915-488c-adb5-8ae1444d3a8c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.120439 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.121145 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.121469 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.121889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.131217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.132692 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5f78-0f6b-4684-a295-c7f68c30f0a6 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.142931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1400/2794] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:09.424168 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.425639 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:09.437932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad9814-173f-4246-bfbc-256ba107521d None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.442719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.443595 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.443890 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.444197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] PATH_INFO: `/v3/projects/940e041b80564c1f92090a3bd98a939e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.445435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940e041b80564c1f92090a3bd98a939e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.447368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.495807 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940e041b80564c1f92090a3bd98a939e', 'request_id': 'req-27977a22-56c3-495c-87d0-3def65f67346'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:09.496943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:09.503789 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940e041b80564c1f92090a3bd98a939e', 'request_id': 'req-27977a22-56c3-495c-87d0-3def65f67346'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:09.510143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940e041b80564c1f92090a3bd98a939e', 'request_id': 'req-27977a22-56c3-495c-87d0-3def65f67346'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:09.522746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] The token cache is being invalidate because project 940e041b80564c1f92090a3bd98a939e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:09.523034 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:09.523668 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-27977a22-56c3-495c-87d0-3def65f67346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:09.525244 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1395/2795] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:09 2022] DELETE /identity/v3/projects/940e041b80564c1f92090a3bd98a939e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:09.530889 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.531188 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.531498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.549389 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:09.550512 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:09.610646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1401/2796] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:09.618488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.618787 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.619073 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.638559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:09.639771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:09.647721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1396/2797] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:09.651950 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.661611 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52075f-14d2-4406-9677-be2536ae703e None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.666228 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.666995 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.667342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.667661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.668391 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:09.668736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.670356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.681675 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1402/2798] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:09.686920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.696255 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c9ffc-5cbe-4409-9613-700cb41b5de6 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.701829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.702479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.702764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.703045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.703751 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.705695 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.731966 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1397/2799] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:09 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:09.736115 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.746813 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b3297-710e-4a08-98c1-df858e4dcfb4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.786698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:09.805030 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.805702 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.806266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.806266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.806951 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.808944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.828052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1403/2800] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:09 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:09.832144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.842934 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc2595-2593-4b82-836a-4514227d91c3 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.848114 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.848804 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.849089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.849368 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.850091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.851693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.863991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1398/2801] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:09.867925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.877870 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc6d1-2230-43cd-8c69-ac9771861605 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.882501 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.883148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.883435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.883715 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.884436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.886149 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5519a-cc2c-4d1f-a90b-6252ff6c4e51 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.896214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1404/2802] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:09.900089 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:09.910580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2788-ebb4-4878-9385-b341399084ed admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:09.915166 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:09.916024 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.916303 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.916609 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] PATH_INFO: `/v3/projects/dbb93cefd22b4ec9915b8e6b134f1f24/users/d088497fd7c84ff4bbac958b681001fa/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.934233 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb93cefd22b4ec9915b8e6b134f1f24, user_id=d088497fd7c84ff4bbac958b681001fa, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:09.936946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:09.950404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1399/2803] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:09 2022] PUT /identity/v3/projects/dbb93cefd22b4ec9915b8e6b134f1f24/users/d088497fd7c84ff4bbac958b681001fa/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:09.961823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:09.962122 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:09.962407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:09.990392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] MFA Rules not processed for user `d088497fd7c84ff4bbac958b681001fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:09.991674 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:10.055533 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1405/2804] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:10.060471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.124197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46598344-fcd8-4f2f-8068-f2ddd5ea336c admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.128859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.129517 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.129801 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.130080 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.130772 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.132721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.157653 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1400/2805] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:10 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:10.161784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.173123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f273d3-cfdd-47c1-8127-9aae63ff26f2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.230526 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.231198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.231484 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.231764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.232487 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.234418 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.254672 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1406/2806] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:10 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:10.258655 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.270056 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a60aff-bfe5-4f58-b91b-1f0d69c24866 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.274722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.275466 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.275749 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.276097 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.277326 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.278927 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.289329 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1401/2807] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:10.293953 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.303836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a05110-8007-46a4-9fd4-5dbcf2635538 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.309235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.310042 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.310326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.310602 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] PATH_INFO: `/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5/users/920ed95a757e42d991b3167d9b0d8f1b/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.325502 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab2703e3a9343d48e7ddccf6f06caf5, user_id=920ed95a757e42d991b3167d9b0d8f1b, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.328177 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.341412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1407/2808] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:10 2022] PUT /identity/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5/users/920ed95a757e42d991b3167d9b0d8f1b/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:10.345414 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.356339 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8e86-a7dc-4503-be92-2eebe812cd91 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.400358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:10.414620 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.415290 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.415577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.415859 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.416619 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.418943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58382c-3737-44c5-bfd9-c7b93e920719 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.428752 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1402/2809] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:10.432979 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.443503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51468892-9697-4fc7-8e29-283f6d8e2a16 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.448192 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.449229 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.449514 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.449788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] PATH_INFO: `/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5/users/920ed95a757e42d991b3167d9b0d8f1b/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.451764 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab2703e3a9343d48e7ddccf6f06caf5, user_id=920ed95a757e42d991b3167d9b0d8f1b, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.454451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.465586 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1408/2810] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:10 2022] PUT /identity/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5/users/920ed95a757e42d991b3167d9b0d8f1b/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:10.476010 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.476300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.476614 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.503449 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] MFA Rules not processed for user `920ed95a757e42d991b3167d9b0d8f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:10.504684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:10.573144 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1403/2811] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:10.584312 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.586053 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:10.591916 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:10.646584 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97713a34-5a6b-4ae7-8249-20f512360650 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.650315 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.650976 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.651797 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.652098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.661217 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.662662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.672641 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1409/2812] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:10.743957 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:10.753721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff67a62-e73d-457b-900d-eff928dc7735 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:10.757388 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:10.758026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:10.758312 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:10.758590 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:10.814393 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:10.815840 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22c843-0f98-403e-898f-b5db1b5e1340 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:10.825562 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1404/2813] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:20.614103 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:20.674791 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77104e32-9710-4816-9779-960f78b92792 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:20.679219 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:20.680079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:20.680367 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:20.680743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] PATH_INFO: `/v3/users/d088497fd7c84ff4bbac958b681001fa` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:20.681932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d088497fd7c84ff4bbac958b681001fa)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:20.684121 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:20.728323 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd088497fd7c84ff4bbac958b681001fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:20.736817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd088497fd7c84ff4bbac958b681001fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:20.737436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd088497fd7c84ff4bbac958b681001fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:20.749610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd088497fd7c84ff4bbac958b681001fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:20.750237 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd088497fd7c84ff4bbac958b681001fa'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:20.762680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1410/2814] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:20 2022] DELETE /identity/v3/users/d088497fd7c84ff4bbac958b681001fa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:20.767183 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:20.767715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:20.767815 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:20.784886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:20.786072 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:20.845214 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1405/2815] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:20.855120 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:20.857895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:20.887334 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:20.918721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8488dd-25bd-4c26-b963-3a553fe5cf65 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:20.922411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:20.923062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:20.923349 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:20.923630 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:20.932720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:20.934691 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:20.944153 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1411/2816] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:21.206847 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:21.208959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:21.227525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bc4cc-3ba2-42bc-acae-479c3cd286ca None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:21.234782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:21.236078 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:21.236546 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:21.236973 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] PATH_INFO: `/v3/projects/dbb93cefd22b4ec9915b8e6b134f1f24` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:21.238771 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb93cefd22b4ec9915b8e6b134f1f24)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:21.241210 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:21.244722 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=100 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:21.293553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb93cefd22b4ec9915b8e6b134f1f24', 'request_id': 'req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.299593 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb93cefd22b4ec9915b8e6b134f1f24', 'request_id': 'req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.305267 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb93cefd22b4ec9915b8e6b134f1f24', 'request_id': 'req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.317370 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] The token cache is being invalidate because project dbb93cefd22b4ec9915b8e6b134f1f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:21.317660 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.318287 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.319836 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1406/2817] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:21 2022] DELETE /identity/v3/projects/dbb93cefd22b4ec9915b8e6b134f1f24 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:21.323941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:21.333406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3cb9e-6272-4d45-a33f-7ec6ed8726ae service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:21.391377 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:21.392275 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:21.392595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:21.392876 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] PATH_INFO: `/v3/users/920ed95a757e42d991b3167d9b0d8f1b` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:21.394051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920ed95a757e42d991b3167d9b0d8f1b)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:21.396045 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:21.442687 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920ed95a757e42d991b3167d9b0d8f1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.450204 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920ed95a757e42d991b3167d9b0d8f1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.450766 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920ed95a757e42d991b3167d9b0d8f1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.462799 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920ed95a757e42d991b3167d9b0d8f1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.463435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-45541137-5d23-423d-a9cf-a9fc2e3ae821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920ed95a757e42d991b3167d9b0d8f1b'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.476384 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1412/2818] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:21 2022] DELETE /identity/v3/users/920ed95a757e42d991b3167d9b0d8f1b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:21.750538 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:21.752046 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:21.766641 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dffc6-0f1c-4fbc-bf0a-0c8d46ca3bd4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:21.826328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:21.827185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:21.827471 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:21.827748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] PATH_INFO: `/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:21.828953 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab2703e3a9343d48e7ddccf6f06caf5)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:21.830874 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:21.879628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab2703e3a9343d48e7ddccf6f06caf5', 'request_id': 'req-0c0fcf3e-2210-4025-9224-8e03343af5c9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.885724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab2703e3a9343d48e7ddccf6f06caf5', 'request_id': 'req-0c0fcf3e-2210-4025-9224-8e03343af5c9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.891963 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab2703e3a9343d48e7ddccf6f06caf5', 'request_id': 'req-0c0fcf3e-2210-4025-9224-8e03343af5c9'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.905252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] The token cache is being invalidate because project 0ab2703e3a9343d48e7ddccf6f06caf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:21.905541 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.906184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-0c0fcf3e-2210-4025-9224-8e03343af5c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:21.907738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1407/2819] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:21 2022] DELETE /identity/v3/projects/0ab2703e3a9343d48e7ddccf6f06caf5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:21.912665 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:21.913171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:21.913290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:21.940970 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:21.942154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:21.975252 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:22.004046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1413/2820] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:24:22.012630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.012923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.013203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.032929 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:22.034119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:22.042241 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1408/2821] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:22.046403 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.055833 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5438c1-5e02-4360-96b5-0633ff85777f None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.061282 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.061926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.062209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.062485 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.063202 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:22.063508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.065180 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.077293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1414/2822] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:22.081343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.090665 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f856e5-3f3c-470d-8ee7-304895da07f7 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.095483 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.096129 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.096437 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.096769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.097769 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.099644 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.112886 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:22.127435 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1409/2823] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:22.131587 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.142110 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841060b-b5a6-42fb-8503-207fe3a9814c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.197421 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.198124 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.198413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.198721 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.199984 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.202091 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.221666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1415/2824] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:22.226042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.236510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d851d-fb83-4420-8d98-98412927d228 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.241062 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.241714 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.241996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.242273 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.242995 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.244688 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.255710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1410/2825] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:22.259610 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.271101 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0949df2-2bb3-4576-90d5-60a7b0a335fc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.275633 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.276281 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.276597 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.276875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.277874 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.279506 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fbd1a-9261-419a-9952-a01bef34e3f6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.289356 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1416/2826] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:22.293626 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.303554 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4fc3b-0cb4-4eb5-91c9-6217fa48560e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.308103 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.309553 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.309834 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.310104 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] PATH_INFO: `/v3/projects/260d4ad6c31f40338a7064bdb058b284/users/eaa5745b77624016b63609c27617b6e4/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.325067 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260d4ad6c31f40338a7064bdb058b284, user_id=eaa5745b77624016b63609c27617b6e4, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.327721 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.340413 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1411/2827] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:22 2022] PUT /identity/v3/projects/260d4ad6c31f40338a7064bdb058b284/users/eaa5745b77624016b63609c27617b6e4/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:22.351652 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.351948 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.352233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.381676 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] MFA Rules not processed for user `eaa5745b77624016b63609c27617b6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:22.382927 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:22.446573 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1417/2828] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:22.451223 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.512573 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0f667-f4e4-438d-a06f-406f59a3a2e2 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.517221 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.517900 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.518184 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.518459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.519156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.521102 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.546654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1412/2829] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/projects => generated 361 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:22.550716 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.562395 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd5b39-6d68-435a-a6d8-dbbbde61cc11 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.591898 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:22.619661 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.620322 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.620638 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.621451 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.622267 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.624203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.643789 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1418/2830] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:22.647780 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.659105 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e270285-711e-4adb-9339-a18365f653c1 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.663946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.664650 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.665053 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.665333 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.666060 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.667846 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.678925 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1413/2831] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:22.682975 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.692907 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682df69f-37dc-4717-ad99-f1d93837481f admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.697540 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.698413 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.698698 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.699498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] PATH_INFO: `/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587/users/e0ca4c4c94554aa0ab7a970d4eb61cd5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.713926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b57d1dcfcd44aaa95e89ba93902587, user_id=e0ca4c4c94554aa0ab7a970d4eb61cd5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.717212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.729862 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1419/2832] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:22 2022] PUT /identity/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587/users/e0ca4c4c94554aa0ab7a970d4eb61cd5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:22.734054 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.744740 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10da34f-c793-418a-a213-232696ad5b00 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.802676 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.803358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.803643 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.803924 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.804678 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.806277 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666861-f76e-48cc-8c0e-d930ae3ffdc6 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.816647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1414/2833] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:22.820827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.832009 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554eb1-1804-41bc-9792-c4904b5820c5 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:22.836574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:22.837446 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.837726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.838000 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] PATH_INFO: `/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587/users/e0ca4c4c94554aa0ab7a970d4eb61cd5/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.840933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b57d1dcfcd44aaa95e89ba93902587, user_id=e0ca4c4c94554aa0ab7a970d4eb61cd5, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:22.843562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:22.853917 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1420/2834] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:22 2022] PUT /identity/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587/users/e0ca4c4c94554aa0ab7a970d4eb61cd5/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:22.864664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:22.865167 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:22.865299 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:22.893248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] MFA Rules not processed for user `e0ca4c4c94554aa0ab7a970d4eb61cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:22.894459 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:22.910918 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:22.963091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1415/2835] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:22.974435 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:22.976235 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:23.033777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46a10-e127-44a8-9b52-10ce8903290d admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:23.037560 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:23.038197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:23.038483 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:23.038765 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:23.047722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:23.049190 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:23.059280 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1421/2836] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:23.132237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:23.142328 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a1464-30e3-47c5-80ae-a58e59d83540 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:23.146213 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:23.146860 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:23.147217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:23.147500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:23.210021 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:23.211469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44228b-5df6-4682-9d31-f02349669a59 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:23.220512 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1416/2837] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:25.910261 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:25.974903 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef23167-359d-43cc-9b72-ba85df8af08b service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:25.979464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:25.980386 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:25.980731 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:25.981004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] PATH_INFO: `/v3/users/eaa5745b77624016b63609c27617b6e4` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:25.982187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa5745b77624016b63609c27617b6e4)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:25.984109 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:26.017795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:26.032271 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa5745b77624016b63609c27617b6e4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.039387 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa5745b77624016b63609c27617b6e4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.040027 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa5745b77624016b63609c27617b6e4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.052143 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa5745b77624016b63609c27617b6e4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.052749 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa5745b77624016b63609c27617b6e4'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.065856 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1422/2838] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:25 2022] DELETE /identity/v3/users/eaa5745b77624016b63609c27617b6e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:26.070417 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:26.070885 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:26.070984 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:26.088791 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:26.090559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:26.149974 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1417/2839] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:26.159600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:26.161036 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:26.221508 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f71a45-ebf6-4dd0-8c06-8dece768e6fc admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:26.225643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:26.226321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:26.226643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:26.226925 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:26.236795 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:26.238290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:26.248701 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1423/2840] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:26.509707 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:26.511142 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:26.523406 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeaaf7e-1b40-4aa4-9cd0-34aa2b750c00 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:26.529013 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:26.529898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:26.530187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:26.530469 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] PATH_INFO: `/v3/projects/260d4ad6c31f40338a7064bdb058b284` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:26.531681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=260d4ad6c31f40338a7064bdb058b284)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:26.533633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:26.579027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:26.580237 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260d4ad6c31f40338a7064bdb058b284', 'request_id': 'req-3cb2536f-55c7-41ba-b20e-f64e6380e1de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.587203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260d4ad6c31f40338a7064bdb058b284', 'request_id': 'req-3cb2536f-55c7-41ba-b20e-f64e6380e1de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.593831 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260d4ad6c31f40338a7064bdb058b284', 'request_id': 'req-3cb2536f-55c7-41ba-b20e-f64e6380e1de'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.606679 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] The token cache is being invalidate because project 260d4ad6c31f40338a7064bdb058b284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:26.606967 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.607589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.609169 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1418/2841] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:26 2022] DELETE /identity/v3/projects/260d4ad6c31f40338a7064bdb058b284 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:26.613187 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:26.623096 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8609-7082-491a-b4aa-3d6fc3a9ff1d service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:26.679532 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:26.680406 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:26.680720 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:26.681002 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] PATH_INFO: `/v3/users/e0ca4c4c94554aa0ab7a970d4eb61cd5` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:26.682186 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ca4c4c94554aa0ab7a970d4eb61cd5)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:26.684207 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:26.728298 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ca4c4c94554aa0ab7a970d4eb61cd5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.733080 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:26.737068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ca4c4c94554aa0ab7a970d4eb61cd5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.738226 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ca4c4c94554aa0ab7a970d4eb61cd5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.751436 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ca4c4c94554aa0ab7a970d4eb61cd5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.752076 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-86947eeb-6214-45aa-bef5-b05bc2f5dc27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ca4c4c94554aa0ab7a970d4eb61cd5'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:26.765181 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1424/2842] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:26 2022] DELETE /identity/v3/users/e0ca4c4c94554aa0ab7a970d4eb61cd5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:27.044378 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.045935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:27.060996 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2536f-55c7-41ba-b20e-f64e6380e1de admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.121340 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.122122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.122411 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.122687 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] PATH_INFO: `/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.123861 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b57d1dcfcd44aaa95e89ba93902587)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.125863 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.173224 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b57d1dcfcd44aaa95e89ba93902587', 'request_id': 'req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:27.179225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b57d1dcfcd44aaa95e89ba93902587', 'request_id': 'req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:27.184932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b57d1dcfcd44aaa95e89ba93902587', 'request_id': 'req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:27.197225 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] The token cache is being invalidate because project b2b57d1dcfcd44aaa95e89ba93902587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:27.197516 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:27.198132 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-c28c06c2-8ab9-4397-bc8c-dc3c7608c041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:27.200496 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1419/2843] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:27 2022] DELETE /identity/v3/projects/b2b57d1dcfcd44aaa95e89ba93902587 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:24:27.206290 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.206794 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.206895 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.226354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:27.227595 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:27.292197 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1425/2844] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:27.301628 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.301920 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.302197 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.320758 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:27.321962 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:27.330172 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1420/2845] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:27.334371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.344755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1248b61-71e8-4513-a095-8e9ad5a2f3bb None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.349593 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.350240 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.350525 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.350802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.351521 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:27.351829 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.353475 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.365639 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1426/2846] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:27.369819 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.380152 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec70b75-d420-4197-b9d8-4087a47a91fe None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.385289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.386032 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.386352 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.386811 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.387320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.389315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.419764 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1421/2847] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:27.421342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.431336 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a238e-a639-4c45-bbd4-750ecfbbf945 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.488066 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.488768 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.489052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.489342 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.490026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.491962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.512390 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1427/2848] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:27.516587 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.527889 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75bd3-01ba-4157-a411-c0ba4793281b admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.532612 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.533268 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.533550 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.533828 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.534549 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.536149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.547390 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1422/2849] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:27.551168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.561402 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8564646-d652-4bb7-ba02-844047ede807 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.566062 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.566756 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.567118 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.567440 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.568170 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.569875 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372058b2-46ed-435e-8813-c48c850c5957 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.580544 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1428/2850] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:27.584586 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.595251 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b6257-fed4-4774-859e-53b4db7ca5a7 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.600307 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.601203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.601485 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.601759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] PATH_INFO: `/v3/projects/b0bded3d7b3749e8adb20fdff3364f04/users/83e9abb852fb4aa5a8164bac8dd1fe85/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.616424 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0bded3d7b3749e8adb20fdff3364f04, user_id=83e9abb852fb4aa5a8164bac8dd1fe85, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.619081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.632122 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1423/2851] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:27 2022] PUT /identity/v3/projects/b0bded3d7b3749e8adb20fdff3364f04/users/83e9abb852fb4aa5a8164bac8dd1fe85/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:24:27.642941 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.643227 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.643505 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.671022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] MFA Rules not processed for user `83e9abb852fb4aa5a8164bac8dd1fe85`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:27.672279 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:27.695617 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:27.738603 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1429/2852] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:27.743110 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.747898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:27.806568 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cf828-9380-4aec-a941-c128147769ea admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.812391 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.813143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.813427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.813705 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.814404 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.816305 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.841343 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1424/2853] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:27.845464 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.857095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d0efa-0b12-4367-a1df-1f1f4dc2c7d9 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.912305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.912926 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.913210 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.913488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.914171 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.916095 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.937944 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1430/2854] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:27 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:27.943051 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.953975 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf488-9b69-4bdb-bb83-db9dd1633dd9 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.958686 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.959344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.959633 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.959935 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:27.960696 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:27.962454 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:27.974580 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1425/2855] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:27.978878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:27.989575 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e14b4d-3238-4116-859d-d5f6d0cef07b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:27.994262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:27.995133 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:27.995422 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:27.995703 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] PATH_INFO: `/v3/projects/3b81e99214b0499f84629738ee8d531e/users/f4eb26236e304c3f89f5e0e13d69ba98/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.010536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b81e99214b0499f84629738ee8d531e, user_id=f4eb26236e304c3f89f5e0e13d69ba98, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:28.013253 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:28.025645 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1431/2856] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:27 2022] PUT /identity/v3/projects/3b81e99214b0499f84629738ee8d531e/users/f4eb26236e304c3f89f5e0e13d69ba98/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:28.030037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:28.040091 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdd37b-f963-4ef4-9785-a70a24828c2f admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:28.097479 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:28.098149 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:28.098436 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:28.098719 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.099441 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:28.101138 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e23a1-8005-42ef-b173-b92871f05fac admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:28.111792 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1426/2857] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:24:28.116051 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:28.126454 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc988d3-a6cc-424e-af3e-a2ce6e43c198 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:28.131148 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:28.132020 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:28.132306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:28.132666 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] PATH_INFO: `/v3/projects/3b81e99214b0499f84629738ee8d531e/users/f4eb26236e304c3f89f5e0e13d69ba98/roles/bde34f992bf3449ca0097f2d900a7537` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.134654 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b81e99214b0499f84629738ee8d531e, user_id=f4eb26236e304c3f89f5e0e13d69ba98, role_id=bde34f992bf3449ca0097f2d900a7537)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:28.137851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:28.147960 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1432/2858] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:28 2022] PUT /identity/v3/projects/3b81e99214b0499f84629738ee8d531e/users/f4eb26236e304c3f89f5e0e13d69ba98/roles/bde34f992bf3449ca0097f2d900a7537 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:28.158528 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:28.159027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:28.159137 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.185746 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] MFA Rules not processed for user `f4eb26236e304c3f89f5e0e13d69ba98`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:28.186864 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:28.253314 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:28.259841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1427/2859] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:28.271350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:28.273239 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:28.353553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0be37-719d-4443-ae84-e46018a77f3c admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:28.357568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:28.358214 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:28.358498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:28.358775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.367669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:28.369154 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:28.374079 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:28.383262 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1433/2860] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:28.487432 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:28.498325 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d525b52-a903-466f-8247-471ab98e1088 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:28.501943 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:28.502591 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:28.502909 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:28.503185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:28.560203 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:28.561693 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd7f60-21c3-4128-b00c-225a02be3ab4 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:28.570822 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1428/2861] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:31.143627 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:31.205725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242a074-c5f1-4490-9d60-f01fe6f10748 service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:31.210225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:31.211089 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:31.211371 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:31.211644 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] PATH_INFO: `/v3/users/83e9abb852fb4aa5a8164bac8dd1fe85` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:31.212852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e9abb852fb4aa5a8164bac8dd1fe85)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:31.215350 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:31.259739 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e9abb852fb4aa5a8164bac8dd1fe85'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.266891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e9abb852fb4aa5a8164bac8dd1fe85'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.267481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e9abb852fb4aa5a8164bac8dd1fe85'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.281054 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e9abb852fb4aa5a8164bac8dd1fe85'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.281725 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e9abb852fb4aa5a8164bac8dd1fe85'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.294400 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1434/2862] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:31 2022] DELETE /identity/v3/users/83e9abb852fb4aa5a8164bac8dd1fe85 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:31.298936 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:31.299425 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:31.299539 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:31.316621 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:31.317799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:31.377508 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1429/2863] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:31.388004 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:31.389438 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:31.446965 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b12a0-57e9-4eb4-b4d2-8817f8796073 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:31.451313 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:31.451962 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:31.452246 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:31.452562 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:31.461574 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:31.463046 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:31.473678 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1435/2864] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:31.750076 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:31.751601 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:31.764919 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152735f-6b4c-4b82-be44-e133466ef6b3 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:31.769938 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:31.771000 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:31.771357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:31.771715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] PATH_INFO: `/v3/projects/b0bded3d7b3749e8adb20fdff3364f04` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:31.773156 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0bded3d7b3749e8adb20fdff3364f04)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:31.775354 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:31.821499 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0bded3d7b3749e8adb20fdff3364f04', 'request_id': 'req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.827959 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0bded3d7b3749e8adb20fdff3364f04', 'request_id': 'req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.833532 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0bded3d7b3749e8adb20fdff3364f04', 'request_id': 'req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.845898 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] The token cache is being invalidate because project b0bded3d7b3749e8adb20fdff3364f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:31.846187 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.846816 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.848654 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1430/2865] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:31 2022] DELETE /identity/v3/projects/b0bded3d7b3749e8adb20fdff3364f04 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:31.852643 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:31.862924 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772ff1-5592-4adf-875b-50d17fdcb605 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:31.881568 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:31.922296 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:31.923172 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:31.923482 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:31.923762 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] PATH_INFO: `/v3/users/f4eb26236e304c3f89f5e0e13d69ba98` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:31.924971 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4eb26236e304c3f89f5e0e13d69ba98)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:31.926943 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:31.975621 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4eb26236e304c3f89f5e0e13d69ba98'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.983189 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4eb26236e304c3f89f5e0e13d69ba98'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.983817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4eb26236e304c3f89f5e0e13d69ba98'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.995858 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4eb26236e304c3f89f5e0e13d69ba98'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:31.996523 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-dbf61881-9b2a-4775-97d2-64b39c503644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4eb26236e304c3f89f5e0e13d69ba98'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.008552 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1436/2866] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:31 2022] DELETE /identity/v3/users/f4eb26236e304c3f89f5e0e13d69ba98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:24:32.287664 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.289232 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:32.304357 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91f53d-4a9a-43b7-8fcb-aebe405802b5 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.322351 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:32.365141 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.365997 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.366285 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.366566 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] PATH_INFO: `/v3/projects/3b81e99214b0499f84629738ee8d531e` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.367748 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b81e99214b0499f84629738ee8d531e)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.369703 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.416427 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b81e99214b0499f84629738ee8d531e', 'request_id': 'req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.423011 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b81e99214b0499f84629738ee8d531e', 'request_id': 'req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.428876 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b81e99214b0499f84629738ee8d531e', 'request_id': 'req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.441964 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] The token cache is being invalidate because project 3b81e99214b0499f84629738ee8d531e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:32.442261 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.442893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-d2823f5e-7acd-456b-ac48-9a7376d0d8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:32.444462 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1431/2867] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:32 2022] DELETE /identity/v3/projects/3b81e99214b0499f84629738ee8d531e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:32.449910 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.450409 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.450510 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.470613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:32.471816 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:32.532429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1437/2868] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:32.592025 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e145ac-b43a-426a-9932-d4092319080a None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.592315 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e145ac-b43a-426a-9932-d4092319080a None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.592636 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e145ac-b43a-426a-9932-d4092319080a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.611384 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-28e145ac-b43a-426a-9932-d4092319080a None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:32.612697 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-28e145ac-b43a-426a-9932-d4092319080a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:32.621313 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1432/2869] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:24:32.625480 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.635141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848a2d2-7b8e-4c49-b5e5-93eb74c18fc2 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.640503 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.641157 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.641445 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.641722 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.642447 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:32.642755 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.644366 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.655841 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1438/2870] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:32.660139 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e145ac-b43a-426a-9932-d4092319080a None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.669525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e145ac-b43a-426a-9932-d4092319080a None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.674158 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.674802 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.675084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.675360 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.676086 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.678016 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.703157 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1433/2871] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:32 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:32.707507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.718225 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e593e0-e7b6-45c1-92ff-b144db1bab09 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.744212 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:32.775741 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.776412 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.776729 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.777549 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.778308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.780232 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.799530 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1439/2872] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:32 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:32.803604 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.814297 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0504e41-2bbf-498d-8feb-22ec85fdab95 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.818799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.819447 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.819728 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.820001 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.820776 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.822402 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.833521 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1434/2873] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:32.837354 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.848106 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e2522-2982-4373-b039-d2631c20b2bb admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.852836 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.853489 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.853777 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.854059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.854775 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.856518 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b103f-d957-40f7-9b18-07d52d82412c admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.866385 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1440/2874] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:32.870667 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:32.881084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fef1f3-90c7-42b8-ae8c-ef2ee971b71e admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:32.885738 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:32.887270 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.887741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.888026 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] PATH_INFO: `/v3/projects/78cbeb7016d140ce9303f60f95d462ce/users/77cf0de7c8034ca1b3b594808a17c4d6/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.903559 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78cbeb7016d140ce9303f60f95d462ce, user_id=77cf0de7c8034ca1b3b594808a17c4d6, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:32.906331 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:32.919071 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1435/2875] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:32 2022] PUT /identity/v3/projects/78cbeb7016d140ce9303f60f95d462ce/users/77cf0de7c8034ca1b3b594808a17c4d6/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:32.928969 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:32.929209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:32.929493 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:32.957681 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] MFA Rules not processed for user `77cf0de7c8034ca1b3b594808a17c4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:32.958891 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:33.023260 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1441/2876] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:33.035005 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:33.036724 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:33.088887 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:33.095983 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a248fb-83ab-454d-9342-9d84bc764f97 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:33.100673 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:33.101321 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:33.101627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:33.101891 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:33.110257 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:33.111690 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b9003-e054-4df0-a284-d737c2cedb4f service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:33.120605 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1436/2877] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:33.140553 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:33.203113 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772eabf-adc0-415a-9399-b27ac465be45 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:33.207732 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:33.208646 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:33.208933 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:33.209211 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] PATH_INFO: `/v3/users/77cf0de7c8034ca1b3b594808a17c4d6` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:33.210405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cf0de7c8034ca1b3b594808a17c4d6)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:33.212382 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:33.253802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:33.261068 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cf0de7c8034ca1b3b594808a17c4d6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:33.269028 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cf0de7c8034ca1b3b594808a17c4d6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:33.269753 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cf0de7c8034ca1b3b594808a17c4d6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:33.282038 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cf0de7c8034ca1b3b594808a17c4d6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:33.282700 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cf0de7c8034ca1b3b594808a17c4d6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:33.295040 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1442/2878] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:33 2022] DELETE /identity/v3/users/77cf0de7c8034ca1b3b594808a17c4d6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:33.299534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:33.299830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:33.300113 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:33.317551 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:33.318750 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:33.379510 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1437/2879] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:33.389203 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:33.390705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:33.448372 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365b78-ed56-49dc-b756-90ac14ebf0c4 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:33.452305 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:33.452974 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:33.453308 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:33.453531 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:33.462331 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:33.463761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc447a1-a989-47cf-a60d-6bbbdef06ee4 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:33.473805 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1443/2880] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:33.992857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:33.994276 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.007491 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3b8d7-3d0b-416c-b8ba-e71dd017ec54 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.013170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.014042 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.014320 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.014599 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] PATH_INFO: `/v3/projects/78cbeb7016d140ce9303f60f95d462ce` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.015830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78cbeb7016d140ce9303f60f95d462ce)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.017782 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.064363 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78cbeb7016d140ce9303f60f95d462ce', 'request_id': 'req-3e93e419-71d9-4bb1-93c8-e7549753827b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.070434 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78cbeb7016d140ce9303f60f95d462ce', 'request_id': 'req-3e93e419-71d9-4bb1-93c8-e7549753827b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.076569 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78cbeb7016d140ce9303f60f95d462ce', 'request_id': 'req-3e93e419-71d9-4bb1-93c8-e7549753827b'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.088177 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] The token cache is being invalidate because project 78cbeb7016d140ce9303f60f95d462ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:34.088492 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.089109 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-3e93e419-71d9-4bb1-93c8-e7549753827b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.091410 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1438/2881] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:33 2022] DELETE /identity/v3/projects/78cbeb7016d140ce9303f60f95d462ce => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:34.096257 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.096596 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.096879 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.115443 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:34.116747 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.178554 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1444/2882] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:34.186870 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.187202 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.187488 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.206382 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:34.207589 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.215645 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1439/2883] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:34.219814 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.229696 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4154af4-3e96-4f96-a52b-7b695475c048 None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.234407 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.235064 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.235353 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.235680 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.236360 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80315) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:24:34.236709 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.238327 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.250567 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1445/2884] 10.222.0.41 () {58 vars in 1152 bytes} [Fri May 27 15:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:24:34.254852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.264634 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116094a7-596b-4f70-b748-d44273815dc0 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.270036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.270700 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.270988 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.271271 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.271971 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.274098 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.299908 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1440/2885] 10.222.0.41 () {60 vars in 1152 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:34.305123 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.316233 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a28884-9f77-4899-bc58-0cce583aa1d6 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.361817 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:34.376647 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.377321 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.377604 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.377878 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.378583 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.380548 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.400429 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1446/2886] 10.222.0.41 () {60 vars in 1143 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:24:34.404486 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.414882 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b3a9-5e88-4fd0-bb52-7866465257c4 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.420045 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.420741 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.421033 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.421317 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.422036 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.423642 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.434867 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1441/2887] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:34.438851 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.448956 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06639b-cd9a-43a3-abaf-174500079c28 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.453820 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.454497 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.454779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.455052 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.455790 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.457420 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5461eae-cdc9-4c6e-919c-ee01e3b37b6d admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.471156 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1447/2888] 10.222.0.41 () {58 vars in 1121 bytes} [Fri May 27 15:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:24:34.475577 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.486248 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406bfb7-1b6d-457e-97be-8f8b11a8ba2a admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.491160 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.492064 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.492341 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.492647 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] PATH_INFO: `/v3/projects/0415ed27275349078c515582f931c32c/users/869d5ad3203241b0a6b298af486bf6e6/roles/66fc02452abf40a398e393da3430e5d2` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.507799 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0415ed27275349078c515582f931c32c, user_id=869d5ad3203241b0a6b298af486bf6e6, role_id=66fc02452abf40a398e393da3430e5d2)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.510396 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.523217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1442/2889] 10.222.0.41 () {60 vars in 1482 bytes} [Fri May 27 15:24:34 2022] PUT /identity/v3/projects/0415ed27275349078c515582f931c32c/users/869d5ad3203241b0a6b298af486bf6e6/roles/66fc02452abf40a398e393da3430e5d2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:34.534811 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] REQUEST_METHOD: `POST` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.535107 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.535392 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.562906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.auth.core [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] MFA Rules not processed for user `869d5ad3203241b0a6b298af486bf6e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80315) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:34.564105 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.627973 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1448/2890] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:34.639035 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.640788 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.651525 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:34.700786 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c1f44-3ebb-45af-811e-47493f60e3bd admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.704416 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.705084 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.705372 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.705656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.714009 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.715961 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ace1a-bfc9-41a2-96c7-d4eb88211890 service cinder] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.725006 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1443/2891] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:34.749852 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:34.812507 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385eac9-3db5-4458-acdd-ceac7264d41a None None] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:34.817111 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:34.817981 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.818265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.818541 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] PATH_INFO: `/v3/users/869d5ad3203241b0a6b298af486bf6e6` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.819743 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869d5ad3203241b0a6b298af486bf6e6)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:34.821726 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:34.867613 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869d5ad3203241b0a6b298af486bf6e6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.875208 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869d5ad3203241b0a6b298af486bf6e6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.875850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869d5ad3203241b0a6b298af486bf6e6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.877773 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:34.889931 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869d5ad3203241b0a6b298af486bf6e6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.890600 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869d5ad3203241b0a6b298af486bf6e6'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:34.908268 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1449/2892] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:24:34 2022] DELETE /identity/v3/users/869d5ad3203241b0a6b298af486bf6e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:34.912833 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:34.913126 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:34.913407 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:34.930884 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] MFA Rules not processed for user `bcba1f2f592c4d6cbf88e82920040619`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:34.932088 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:34.992790 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1444/2893] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:35.002569 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:35.003972 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:35.061653 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9bcf-1c02-4148-926c-28f97b33d345 admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:35.065326 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:35.066017 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:35.066306 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:35.066581 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:35.076195 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:35.077662 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:35.087073 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1450/2894] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:35.354374 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:35.356027 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:35.369691 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31903b90-cfec-41d7-be17-8404a6966ade None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:35.378715 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:35.379913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:35.380207 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:35.380522 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] PATH_INFO: `/v3/projects/0415ed27275349078c515582f931c32c` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:35.381829 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0415ed27275349078c515582f931c32c)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:35.383803 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:35.433291 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0415ed27275349078c515582f931c32c', 'request_id': 'req-4b6993fd-8061-4240-9410-0570bba483e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:35.439923 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0415ed27275349078c515582f931c32c', 'request_id': 'req-4b6993fd-8061-4240-9410-0570bba483e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:35.445630 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0415ed27275349078c515582f931c32c', 'request_id': 'req-4b6993fd-8061-4240-9410-0570bba483e7'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:35.458946 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] The token cache is being invalidate because project 0415ed27275349078c515582f931c32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:24:35.459500 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:35.460143 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-4b6993fd-8061-4240-9410-0570bba483e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:24:35.461709 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1445/2895] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:24:35 2022] DELETE /identity/v3/projects/0415ed27275349078c515582f931c32c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:24:43.633295 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:43.634705 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:43.691634 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b6c1-5a06-47e2-aaa0-5e7b6fa5ec95 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:43.739827 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1a532f2e8b446b7a03b38b3dda377a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:43.740498 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:43.740788 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:43.741069 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:43.742864 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:43.787043 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:43.802209 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:43.803657 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:43.813452 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1451/2896] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:45.455800 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:50.641818 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] REQUEST_METHOD: `POST` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:50.642394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:50.642394 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:50.659841 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.auth.core [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] MFA Rules not processed for user `60ff7c679e9747898bbc44da1d885b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80314) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:50.661227 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:50.668217 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1446/2897] 10.222.0.41 () {58 vars in 957 bytes} [Fri May 27 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:24:53.809067 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:00.654464 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=3 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:05.467845 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:05.469761 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.571488 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80a59-1e38-4e6d-b88c-49ffc2f417be service cinder] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:05.578026 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2d44da5ce124bd5b18462136f91fcb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:05.578686 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:05.579007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:05.579236 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:05.581044 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.593823 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:05.595579 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:05.606802 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1452/2898] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:06.047530 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:06.048976 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:06.143857 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1ffbd-9a24-44ff-8b32-2e16bb169430 None None] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:06.149455 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:06.150108 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:06.150397 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:06.150681 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:06.152494 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:06.164913 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:06.166515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:06.177627 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1447/2899] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:09.346481 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:09.348173 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:09.408848 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853da7b2-33c8-4e0d-b687-a371a3670cb9 service nova] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:09.412612 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:09.413266 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:09.413550 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:09.413955 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:09.415779 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:09.480129 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:09.481626 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:09.491141 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1453/2900] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:25:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:16.173140 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=41 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:19.486837 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=87 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:01.355119 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:01.366275 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc31f6-38bd-442b-a9af-2ceae93c0eb2 service placement] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:01.370383 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:01.371461 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:01.371755 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:01.372037 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:01.377344 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:01.404433 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:01.405932 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:01.427656 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1448/2901] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:02.606287 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:02.615993 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d235-1761-432e-a041-0fd7864d9856 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:02.619706 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:02.620513 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:02.620803 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:02.621104 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:02.623007 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:02.637784 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:02.639168 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:02.650648 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1454/2902] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:07.777515 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:07.787087 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1901b2-59d4-41b5-b1eb-028176cf00dc service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:07.790797 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f3ecb0e259e4ed89f354e0e95927f10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:07.791438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] REQUEST_METHOD: `GET` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:07.791726 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:07.792008 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:07.793852 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:07.858617 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:07.860101 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:07.869903 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1449/2903] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:12.645938 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=6 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:14.181932 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:14.185265 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:14.201328 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5ffad-2770-4d98-944d-aad32162de25 service neutron] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:14.206519 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:14.207383 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:14.207663 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:14.207935 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] PATH_INFO: `/v3/users/60ff7c679e9747898bbc44da1d885b36` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:14.209138 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ff7c679e9747898bbc44da1d885b36)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:14.211059 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:14.255906 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ff7c679e9747898bbc44da1d885b36'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.263117 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ff7c679e9747898bbc44da1d885b36'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.263923 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ff7c679e9747898bbc44da1d885b36'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.276536 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ff7c679e9747898bbc44da1d885b36'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.277206 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ff7c679e9747898bbc44da1d885b36'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.290411 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1455/2904] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:27:14 2022] DELETE /identity/v3/users/60ff7c679e9747898bbc44da1d885b36 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:14.469456 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:14.483862 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ff0ae-239c-4c12-9220-ff1f65155350 service neutron] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:14.541635 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:14.542423 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:14.542710 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:14.542991 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] PATH_INFO: `/v3/projects/67116cdacaa846138f7835ac5f53b863` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:14.544170 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67116cdacaa846138f7835ac5f53b863)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:14.546120 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:14.592775 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67116cdacaa846138f7835ac5f53b863', 'request_id': 'req-dbab58fa-dc17-49c9-b0ef-a47a39513931'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.599171 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67116cdacaa846138f7835ac5f53b863', 'request_id': 'req-dbab58fa-dc17-49c9-b0ef-a47a39513931'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.604684 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67116cdacaa846138f7835ac5f53b863', 'request_id': 'req-dbab58fa-dc17-49c9-b0ef-a47a39513931'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.606300 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:14.617893 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] The token cache is being invalidate because project 67116cdacaa846138f7835ac5f53b863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:27:14.618185 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.618830 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.620377 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1450/2905] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:27:14 2022] DELETE /identity/v3/projects/67116cdacaa846138f7835ac5f53b863 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:14.624405 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:14.625912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:14.639912 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490b03-2408-4eb2-b73f-1dd26f01ec7b admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:14.696682 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:14.697563 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:14.697850 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:14.698127 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] PATH_INFO: `/v3/users/8cb6ab6278444aa8875607ad1e5cf19f` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:14.699319 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb6ab6278444aa8875607ad1e5cf19f)` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:14.701293 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:14.746915 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb6ab6278444aa8875607ad1e5cf19f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.754669 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb6ab6278444aa8875607ad1e5cf19f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.755426 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb6ab6278444aa8875607ad1e5cf19f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.767557 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb6ab6278444aa8875607ad1e5cf19f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.768198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.notifications [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb6ab6278444aa8875607ad1e5cf19f'} {{(pid=80315) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:14.770198 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:14.783205 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1456/2906] 10.222.0.41 () {58 vars in 1223 bytes} [Fri May 27 15:27:14 2022] DELETE /identity/v3/users/8cb6ab6278444aa8875607ad1e5cf19f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:14.957081 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Authenticating user token {{(pid=80314) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:14.958637 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.fernet_utils [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80314) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:14.973061 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab58fa-dc17-49c9-b0ef-a47a39513931 admin admin] Validating token access rules against request {{(pid=80314) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:15.031716 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bcba1f2f592c4d6cbf88e82920040619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ab0327c3107460fb9252bf46e1b56cc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80314) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:15.033292 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:15.033759 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:15.034049 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] PATH_INFO: `/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88` {{(pid=80314) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:15.035252 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0da8d3c342f4479a49f5ec0fa361f88)` {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:15.037239 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] RBAC: Authorization granted {{(pid=80314) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:15.083289 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0da8d3c342f4479a49f5ec0fa361f88', 'request_id': 'req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:15.089358 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0da8d3c342f4479a49f5ec0fa361f88', 'request_id': 'req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:15.096234 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0da8d3c342f4479a49f5ec0fa361f88', 'request_id': 'req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d'} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:15.108534 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] The token cache is being invalidate because project e0da8d3c342f4479a49f5ec0fa361f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80314) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:27:15.108824 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:15.109438 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG keystone.notifications [None req-69ebbf7e-eaad-4b7c-a607-2b070db85e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80314) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:15.111575 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: [pid: 80314|app: 0|req: 1451/2907] 10.222.0.41 () {58 vars in 1232 bytes} [Fri May 27 15:27:14 2022] DELETE /identity/v3/projects/e0da8d3c342f4479a49f5ec0fa361f88 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:24.779746 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=4 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:25.105487 dv-cinder-842541-3-iscsi devstack@keystone.service[80314]: DEBUG dbcounter [-] [80314] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=53 {{(pid=80314) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:30:10.929254 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Authenticating user token {{(pid=80315) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:30:10.930671 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:30:11.001751 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734233b-37bd-4f19-a896-565e0c1eb86a admin admin] Validating token access rules against request {{(pid=80315) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:30:11.062086 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a34be8446d14cf1b9a76e9ffd0d4921', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e948f94140fc4057832fe02726946abc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80315) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:30:11.062736 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] REQUEST_METHOD: `GET` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:30:11.063022 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] SCRIPT_NAME: `/identity` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:30:11.063307 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80315) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:30:11.065093 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.fernet_utils [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80315) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:30:11.076175 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:30:11.077819 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36e99f-9d94-4643-a325-5ea50ce21de2 service placement] RBAC: Authorization granted {{(pid=80315) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:30:11.088098 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: [pid: 80315|app: 0|req: 1457/2908] 10.222.0.41 () {60 vars in 1365 bytes} [Fri May 27 15:30:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:30:21.083723 dv-cinder-842541-3-iscsi devstack@keystone.service[80315]: DEBUG dbcounter [-] [80315] Writing DB stats keystone:SELECT=58 {{(pid=80315) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}